Character image processing method and device

Abstract

The invention discloses a character image processing method and device. The method comprises the following steps: acquiring an initial character image to be reconstructed, and determining character contents and character positions corresponding to character objects in the initial character image; cutting the character object from the initial character image according to the character position to obtain an initial character image corresponding to the character object; and performing reconstruction processing by using the character content and the corresponding initial character image to obtain a reconstructed character image corresponding to the initial character image. According to the scheme, character-level refined reconstruction is realized by combining the bimodal features of the character content and the character image. Through a fusion mechanism of semantic features and visual features, the model can adaptively adjust structure detail weights of different characters in a reconstruction stage, so that the character shape is restored, and meanwhile, the style consistency and the global typesetting stability among the characters are kept.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for text image processing. Background Technology

[0002] With the rapid growth in demand for document digitization and smart office solutions, image capture of documents has been widely adopted in scenarios such as invoice management, document archiving, and mobile office, gradually replacing traditional scanner input methods. However, due to factors such as uneven lighting, camera shake, and limited image quality during the shooting process, the acquired document images often suffer from problems such as blurriness, insufficient clarity, and loss of detail, directly affecting the user's reading experience and the accuracy of subsequent text recognition.

[0003] Existing technologies primarily rely on general image super-resolution algorithms, which enhance image resolution through convolutional reconstruction or feature interpolation. However, these methods are generally designed for natural scenes and lack specific modeling of text structure and semantic features, thus exhibiting significant shortcomings in text image processing. When the text regions in the original image are highly blurred or the character size is small, traditional super-resolution algorithms struggle to achieve fine stroke-level restoration, often resulting in unclear character outlines, stroke overlap, and local artifacts—the common "tadpole script" phenomenon. This type of structural distortion not only weakens the visual readability of the image but also reduces the reliability of OCR recognition, severely limiting the practical application of general super-resolution algorithms in text-intensive scenarios such as document reconstruction, invoice recognition, and video subtitle enhancement. Summary of the Invention

[0004] This application provides a text image processing method and apparatus, solving the problems of insufficient detail restoration capability, character structure distortion, and stroke blurring in existing general super-resolution algorithms. The method identifies the character content and position in a text image, extracts the corresponding character regions, and performs reconstruction processing based on the character content information to generate a reconstructed text image. By introducing character content and character-level image constraints during the reconstruction process, this method achieves accurate restoration of stroke details and maintains consistent overall layout, significantly improving the clarity and readability of the text image. This solution is applicable to various text image scenarios, such as document scan restoration, invoice image enhancement, and low-resolution video subtitle reconstruction.

[0005] Firstly, this application provides a text image processing method, including: Obtain the initial text image to be reconstructed, and determine the character content and character position corresponding to the character objects in the initial text image; Based on the character position, the character object is cropped from the initial text image to obtain the initial character image corresponding to the character object; The reconstructed text image is obtained by using the character content and its corresponding initial character image for reconstruction processing.

[0006] Secondly, this application provides a text image processing apparatus, comprising: The feature extraction module is used to acquire the initial text image to be reconstructed and to determine the character content and character position corresponding to the character objects in the initial text image; A character cropping module is used to crop the character object from the initial text image according to the character position to obtain the initial character image corresponding to the character object; The super-resolution reconstruction module is used to perform reconstruction processing using the character content and its corresponding initial character image to obtain the reconstructed character image corresponding to the initial character image.

[0007] Thirdly, this application provides a text image processing apparatus, comprising: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the text image processing method as described in the first aspect.

[0008] Fourthly, this application provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the text image processing method as described in the first aspect.

[0009] In this application, a text image reconstruction method based on character content recognition and region localization is constructed, achieving structured analysis and accurate reconstruction of input text images. This method acquires the initial text image to be reconstructed and uses a recognition model to determine the character content and position of character objects in the image. Subsequently, based on the character positions, corresponding character images are extracted to form character-level input for reconstruction. On this basis, reconstruction processing is performed by combining the character content and the extracted character images to generate a reconstructed text image that is semantically consistent with the original image and presents a higher quality result. Through the linked processing flow of character parsing, region cropping, and content-driven reconstruction, this application achieves fine-grained reconstruction capabilities for text images, improving the accuracy and structural consistency of image restoration, and is applicable to various application scenarios such as document repair, character enhancement, and image readability improvement. Attached Figure Description

[0010] Figure 1 This is a flowchart of a text image processing method provided in an embodiment of this application; Figure 2 This is a flowchart of a method for determining character content and character region provided in an embodiment of this application; Figure 3 This is a flowchart of a method for determining text content and a first character region provided in an embodiment of this application; Figure 4 This is a flowchart of a text content correction method provided in an embodiment of this application; Figure 5 This is a flowchart of a character position determination method provided in an embodiment of this application; Figure 6 This is a flowchart of a text image reconstruction method provided in an embodiment of this application; Figure 7 This is a flowchart of a priori feature generation method provided in an embodiment of this application; Figure 8 This is a flowchart of an initial image feature map generation method provided in an embodiment of this application; Figure 9 This is a schematic diagram of a text image processing device provided in an embodiment of this application; Figure 10 This is an algorithm framework diagram of a text image processing method provided in an embodiment of this application; Figure 11 This is a structural block diagram of a text image processing apparatus provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of a text image processing device provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as being processed sequentially, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. A process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0012] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0013] Currently, with the rapid development of office automation, document digitization, and intelligent recognition technologies, text image processing tasks are showing a trend towards higher definition, more structured representation, and enhanced readability. However, document images also exhibit significant characteristics of multi-source complexity and unstable quality during acquisition and use. Clearly presenting text content has become a crucial aspect of ensuring a good reading experience, improving OCR recognition accuracy, and enhancing the efficiency of electronic document management. However, in reality, a large number of documents are captured using portable devices such as mobile phones and cameras. While these devices offer advantages such as high flexibility and low cost, they struggle to maintain stable image quality under varying lighting conditions, shooting angles, device imaging quality, and hand-held shakiness. This results in document images that are generally blurry, low-resolution, plagued by noise interference, and lacking structural details, severely impacting the recognition of text content.

[0014] At the engineering application level, most existing document image enhancement methods rely on traditional interpolation magnification or general image super-resolution models. However, these methods cannot effectively model text edge sharpness, character stroke structure, and layout rules, making it difficult to improve resolution while maintaining character detail fidelity. Furthermore, conventional image enhancement algorithms often perform unstably when faced with complex imaging factors such as shadows, reflections, compression artifacts, and image distortion, easily introducing artifacts or causing character structure distortion, further weakening OCR recognition quality. Meanwhile, existing models have limited support for document scene specificity, failing to fully utilize the semantic rules of text regions, character arrangement characteristics, and document layout features, resulting in insufficient super-resolution recovery capabilities and failing to meet the high-precision text recovery requirements in practical business scenarios. Finally, there is currently a lack of a lightweight reconstruction method that can significantly improve document image clarity without adding additional imaging hardware or changing the acquisition process, thus limiting performance improvements in scenarios such as mobile document processing, low-quality image OCR, and electronic document management.

[0015] Therefore, this invention aims to propose a method and apparatus for text image processing. This method enables precise extraction and structured input of text regions by acquiring an initial text image to be reconstructed and accurately analyzing the character content and position of character objects, without relying on additional imaging hardware. Based on this, reconstruction calculations are performed using the character content and its corresponding initial character image, thereby generating a reconstructed text image with higher clarity and better structural coherence. This solution effectively improves the reconstruction effect and readability of low-quality document images, while possessing good versatility and adaptability, meeting the needs for improved text clarity and image quality optimization in various scenarios such as photo scanning, OCR preprocessing, and document digitization.

[0016] Figure 1 A flowchart of a text image processing method provided in an embodiment of this application is given. (Reference) Figure 1 The text image processing method specifically includes: S110. Obtain the initial text image to be reconstructed, and determine the character content and character position corresponding to the character object in the initial text image.

[0017] In some embodiments, an initial text image to be reconstructed is first acquired, whereby the initial text image refers to image data containing original character distribution and visual texture information. A character object refers to an independently locatable character unit identified in the initial text image through local texture differences, edge contours, or connected component features. Character content refers to the specific text content corresponding to the character object at the semantic level, and character position refers to the two-dimensional spatial coordinate information of the character object in the initial text image, such as the position parameters of its bounding box, center point position, or baseline position. After acquiring the initial text image, the character content and character position corresponding to the character object are determined accordingly.

[0018] In one embodiment, the method for determining character content may be: inputting the local visual features of the character object into the OCR recognition model or performing character feature matching on the local visual features of the character object, and outputting the corresponding text content in the pre-trained character dictionary to determine the output text content as the character content.

[0019] In one embodiment, the character position can be determined by: calculating the bounding rectangle, minimum envelope, or baseline offset of the pixel connected region of the character object to generate two-dimensional coordinate information of the character object, and then determining the generated two-dimensional coordinate information as the character position.

[0020] Optionally, Figure 2 A flowchart of a method for determining character content and character region according to an embodiment of this application is provided. (Reference) Figure 2 The method for determining the character content and character region specifically includes: S1101. Input the initial text image into the first trained artificial intelligence model to obtain the character content corresponding to at least one character object in the initial text image and the first character region in which it is located.

[0021] Here, the initial text image refers to text image data containing original character distribution and visual texture information; the character object refers to a independently locatable character unit identified in the initial text image through local texture differences, edge contours, or connected component features; the character content refers to the specific text content corresponding to the character object at the semantic level; the first character region can be the two-dimensional spatial coordinate information of the character object in the initial text image, such as its bounding box, center point position, or baseline position; and the first artificial intelligence model refers to a deep learning model trained to achieve character recognition and localization functions, such as a model built based on convolutional neural networks, visual transformers, or hybrid feature fusion networks.

[0022] In one embodiment, the character content can be obtained by: encoding the visual features of the character objects in the initial text image using a first artificial intelligence model, and using a classification or sequence prediction module to output the text content corresponding to the initial text objects, and then determining the output text content as the character content.

[0023] In one embodiment, the first character region can be determined by: generating the bounding box or feature map coordinates of each character object through the region detection module of the model, mapping them to the two-dimensional spatial position of the initial text image, and determining the mapped two-dimensional spatial position as the first character region.

[0024] In one embodiment, to improve recognition accuracy, the initial text image can be preprocessed before model input, including size normalization, grayscale or color channel normalization, and noise suppression.

[0025] In one embodiment, the first artificial intelligence model can be a CRNN (Convolutional Recurrent Neural Network) model, which is a sequence recognition model based on the combination of CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network). The CNN is used to extract local and global visual features of character objects, while the RNN is used to model the contextual relationships of character sequences to achieve semantic prediction of continuous text. Alternatively, the first artificial intelligence model can employ a visual transformer architecture, using a self-attention mechanism to capture global dependencies between character objects, thereby improving the recognition ability of complex text layouts.

[0026] In one embodiment, the first artificial intelligence model can integrate a multi-scale feature pyramid or a feature fusion module to enhance the detection and recognition capabilities of characters of different sizes, fonts, and rotation angles, thereby achieving higher character positioning accuracy and recognition accuracy in various initial text images.

[0027] Optionally, Figure 3 A flowchart of a method for determining text content and a first character region according to an embodiment of this application is provided. (Reference) Figure 3 The method for determining the text content and the first character region specifically includes: S11011. Input the initial text image into the first trained artificial intelligence model to obtain a character probability distribution sequence arranged according to time steps.

[0028] The initial text image refers to text image data containing original character distribution and visual texture information; the first artificial intelligence model refers to a deep learning model trained to achieve character recognition and probability prediction functions, such as a model based on convolutional neural networks, recurrent neural networks, long short-term memory networks, or visual transformers; the character probability distribution sequence refers to the probability vectors output in the order of time steps, corresponding to the character category at each time step, used to reflect the model's confidence prediction for each character category.

[0029] In one embodiment, the character probability distribution sequence can be obtained by: mapping the visual features of the initial text image to a serialized representation through forward inference of the model, and outputting the probability distribution of each character category through a classifier at each time step, and using the input probability distribution as the character probability distribution sequence.

[0030] In one embodiment, the character probability distribution sequence can be obtained by serializing the probability vectors of each character in the text image according to their spatial order in the image (e.g., from left to right, from top to bottom), thus forming a sequentially arranged sequence of character vectors, i.e., a character probability distribution sequence arranged by time steps. A time step refers to the position index in the sequence, and each time step corresponds to the recognition result of a specific character in the image. During sequence modeling or decoding, the model processes these character vectors one by one according to the time steps, thereby maintaining the consistency of the character order and achieving accurate reconstruction or text output. For example, for an image containing the characters "ABC", the character recognition model obtains the probability vector of each character. , , The sequence is obtained by arranging the steps according to time. , , This sequence can be used for subsequent text decoding or image reconstruction.

[0031] In one embodiment, the initial text image can be preprocessed before model input, including image normalization, resizing, and noise suppression, to improve the accuracy of character probability distribution prediction.

[0032] S11012. Decode the character probability distribution sequence to obtain the character content and time step index range corresponding to at least one character object in the initial text image.

[0033] Here, the character probability distribution sequence refers to the probability vector of each time step corresponding to the character category, output sequentially according to the time steps; decoding refers to the process of converting the probability distribution sequence into a discrete character sequence, including selecting the most likely character category from the probability vector or generating character output through a sequence decoding algorithm; character content refers to the text content corresponding to the character object at the semantic level after decoding; and the time step index range refers to the start and end time steps of the character in the probability distribution sequence, used to identify the position and span of the character in the sequence.

[0034] In one embodiment, the decoding method may be: using a greedy algorithm, beam search, or connection-based temporal classification decoding method to generate character content and its corresponding time step index range from the character probability distribution sequence.

[0035] In one embodiment, the time step index range can be determined by mapping the probability distribution at each time step after spatial sequential serialization to a confidence curve, smoothing the confidence curve, identifying several consecutive high confidence intervals using a threshold segmentation method, and using the start and end time steps of each high confidence interval as the time step index range of the corresponding character.

[0036] For example: Assume the initial text image has a width W = 300 pixels; divide it into T = 30 time steps (each time step ≈ 10 columns of pixels). The confidence sequence obtained at a certain position in the initial text image is: c = [0.02, 0.05, 0.07, 0.40, 0.72, 0.85, 0.78, 0.60, 0.30, 0.10, ...], and the set confidence threshold is... Then the binary sequence is 1 at time steps 5–8, and the index range of the time steps corresponding to the identified characters is [ .

[0037] In one embodiment, the time step index range can be used to project back the initial text image through a linear mapping relationship from time steps to image pixels, thereby obtaining the corresponding spatial start and end columns, i.e., the first text region. To avoid false detections due to noise, strategies such as smoothing filtering, minimum interval length constraints, peak merging, and overlapping interval processing can be employed in practical applications to improve robustness.

[0038] S11013. Map the time step index range to the horizontal region of the initial text image to determine the first character region where at least one character object corresponding to the time step index range is located.

[0039] Among them, the time step index range refers to the start and end time steps of the character in the sequence obtained by decoding the character probability distribution sequence; the horizontal region refers to the continuous pixel segment divided along the horizontal direction in the initial text image, which is used to correspond to the spatial distribution of the time step in the image; the first character region refers to the point-level spatial parameters of the character object in the initial text image obtained after recognizing the character content through the first artificial intelligence model. These point-level spatial parameters can be used to represent the horizontal position of the character in the initial text image for subsequent cropping, reconstruction or enhancement processing.

[0040] In one embodiment, the mapping can be achieved by: establishing a linear mapping relationship between time steps and pixel positions using the width of the initial text image and the total number of time steps in the character probability distribution sequence; mapping the time step index range of the character to the horizontal coordinate interval of the image, thereby obtaining the first character region. For example, the time step index range... The corresponding character range is:

[0041]

[0042] in, The original image width, The length of the feature sequence. For CNN's stride, This represents the range of time step indexes.

[0043] In one embodiment, to improve mapping accuracy, the horizontal region can be fine-tuned or locally expanded by combining the height features of the character object or the overall structure of the image to ensure that the first character region completely covers all strokes of the target character.

[0044] In one embodiment, the mapped first character region can be used as the input region for subsequent character cropping, reconstruction, and visual enhancement processing, ensuring that character processing is accurate and controllable.

[0045] Optionally, Figure 4 A flowchart of a text content correction method provided in an embodiment of this application is given. (Reference) Figure 4 The specific methods for correcting this text content include: S11014. Combine the character content to obtain the text content corresponding to the initial text image.

[0046] Among them, character content refers to the text content corresponding to the character object at the semantic level obtained through the aforementioned decoding steps; combination refers to arranging each character content in sequence according to the spatial order of the characters in the initial text image to form a complete text sequence; text content refers to the continuous text information after the character objects contained in the initial text image are combined according to the spatial layout, which can be used for subsequent text analysis, editing or reconstruction processing.

[0047] In one embodiment, the character content can be combined as follows: according to the position of the first character region of each character object in the image, and based on the sorting rule of horizontal priority and vertical assistance, the character content is sequentially concatenated to obtain the complete text content.

[0048] In one embodiment, character content can be corrected or normalized before combination, including spelling correction, character duplication deletion, or character spacing adjustment, to improve the readability and semantic accuracy of the text content.

[0049] In one embodiment, the combined text content can be used as input to a subsequent reconstruction model, text recognition system, or visual display module to achieve semantic reconstruction and visual enhancement of the initial text image.

[0050] S11015. Perform text correction processing on the text content to obtain the corrected content corresponding to the text content.

[0051] Among them, text content refers to the continuous text sequence obtained by combining character objects in the initial text image; text correction processing refers to the process of detecting and correcting possible recognition errors, missing characters, repeated characters, or non-standard spelling in the text content through algorithms or models; corrected content refers to the text sequence that is semantically complete and has accurate characters after text correction processing, which can be used for subsequent text reconstruction, editing, or semantic analysis.

[0052] In one embodiment, text correction can be achieved by: inputting text content into a language model, using a rule matching algorithm to match the text content or performing contextual analysis on the characters in the text content, and replacing, inserting, or deleting abnormal characters or sequences appearing in the text content to generate corrected content.

[0053] In one embodiment, text correction can be achieved by using NLP (Natural Language Processing) to recognize the text content, correcting erroneously recognized text, and controlling NLP to not add or delete characters, but only perform "substitution correction" on existing characters, thereby making character recognition more accurate. For example, if the recognition result is "There is a great elephant on the African savanna", the corresponding corrected result is "There is an elephant on the African savanna".

[0054] In one embodiment, the correction process may include spell checking, homograph replacement, contextual semantic consistency verification, and character order adjustment to improve the accuracy and readability of the text content.

[0055] In one embodiment, the corrected content can be used as input for subsequent reconstruction of text images, text display, or natural language processing modules to ensure the reliability and semantic consistency of downstream processing.

[0056] S11016. Correct the character content according to the correction content to obtain the corrected character content.

[0057] Among them, the corrected content refers to the semantically complete and character-accurate text sequence obtained through text correction processing; the character content refers to the original character sequence obtained by decoding each character object in the initial text image; the corrected character content refers to the sequence obtained by mapping the corresponding character information in the corrected content to the original character object and replacing or adjusting the erroneous or inconsistent parts in the character content, which is used to ensure the consistency between character-level information and semantic-level information.

[0058] In one embodiment, the correction of character content can be achieved by aligning the characters in the correction content one by one to the original character object order, and performing replacement, insertion, or deletion operations based on the character position or time step index to generate the corrected character content.

[0059] In one embodiment, to improve the accuracy of the correction, the characters in the correction content can be mapped to the corresponding spatial regions in the image by combining the first character region information of the character object, so as to ensure that the corrected character content is consistent with the position, layout and visual features of the initial text image.

[0060] In one embodiment, the corrected character content can be used as input for subsequent character cropping, reconstruction, or text image generation modules, thereby ensuring the semantic accuracy and visual consistency of the reconstructed text image.

[0061] S1102. Input the initial text image into the trained second artificial intelligence model to obtain the second character region corresponding to at least one character object in the initial text image.

[0062] The initial text image refers to text image data containing original character distribution and visual texture information; the second artificial intelligence model refers to a deep learning model trained to achieve character region detection and localization, such as a model based on convolutional neural networks, visual transformers, or hybrid feature fusion networks; the second character region refers to the two-dimensional spatial coordinate information of the character object predicted by the model in the initial text image, such as the bounding box, minimum envelope region, or center point position, which is used to assist in subsequent character cropping, reconstruction, or enhancement processing.

[0063] In one embodiment, the second character region can be obtained by: encoding the visual features of the initial text image using a second artificial intelligence model, outputting the spatial region information of each character object using a region detection or heatmap generation module, and using the output spatial region information as the second character region.

[0064] In one embodiment, the second AI model can be a character detection model based on YOLO (You Only Look Once, a real-time object detection algorithm). YOLO is an object detection algorithm capable of real-time localization and classification of objects in an input image. The YOLO-based character detection model extracts features from the initial text image using an end-to-end convolutional neural network structure and directly regresses the bounding box coordinates and class probability of each character object on the feature map, thus achieving character-level detection. The YOLO-based character detection model can accurately locate each character on a text strip and output the bounding box coordinates of the character in two-dimensional space. To improve detection accuracy, multi-scale feature fusion, data augmentation, and character class balancing strategies can be introduced during the training phase of the character detection model. This allows the model to adapt to character detection tasks with different fonts, sizes, and arrangements, thereby providing accurate character position references for subsequent text reconstruction or sequence recognition.

[0065] In one embodiment, the prediction of the second character region can be combined with confidence threshold filtering, non-maximum suppression, or multi-scale feature fusion to improve the accuracy and completeness of character region localization.

[0066] In one embodiment, the obtained second character region can be used as the input region for subsequent character cropping, feature extraction and reconstruction processing to ensure that subsequent operations maintain the accuracy of the spatial position and boundaries of the character object.

[0067] S1103. Determine the character position corresponding to each character object based on at least one first character region and at least one second character region.

[0068] The first character region can be the point-level spatial parameters of the character object in the initial text image obtained after recognizing the character content through the first artificial intelligence model. These point-level spatial parameters can be used to represent the horizontal position of the character in the initial text image. The second character region can be the region-level spatial parameters of the character object in the initial text image obtained through the second artificial intelligence model. These parameters represent the complete two-dimensional spatial region of the character in the initial text image. The first and second character regions are different precision representations of the character object. The first character region is a low-precision one-dimensional region representation, mainly reflecting the horizontal position and sequence information of the character. It can be mapped to the approximate two-dimensional coordinates of the character in the image and corresponds to the character content information of the character object. The second character region is a high-precision region-level representation, containing the complete two-dimensional boundary information of the character, such as the bounding box or pixel-level mask. It can accurately characterize the spatial range and contour shape of the character, but it does not correspond to the character content information of the character object. The character position can be the final determined spatial region of the character object in the initial text image, such as the bounding box, center point position, or baseline position, used for subsequent character cropping, reconstruction, and text reconstruction processing.

[0069] In one embodiment, the character position can be determined by merging the first character region and the second character region, and generating a character position that comprehensively considers the character content and the region detection results through region overlap calculation, confidence weighting, or bounding box merging algorithms.

[0070] In one embodiment, to improve the accuracy of character position, the merged area can be fine-tuned, for example, by expanding the margins, correcting the aspect ratio, or aligning the baseline, to ensure that the character position fully covers the stroke range of the target character.

[0071] In one embodiment, the determined character position can serve as the basis for subsequent character cropping, reconstruction, text combination, and correction processing, ensuring the consistency between character-level information and image spatial layout.

[0072] Optionally, Figure 5 A flowchart of a character position determination method provided in an embodiment of this application is given. (Reference) Figure 5 The method for determining the position of this character specifically includes: S11031. Calculate the region intersection-union ratio between each pair of first character regions and each second character region.

[0073] The first character region can be the point-level spatial parameter of the character object in the initial text image obtained after recognizing the character content through the first artificial intelligence model. This point-level spatial parameter can be used to represent the horizontal position of the character in the initial text image. The second character region can be the region-level spatial parameter of the character object in the initial text image detected by the second artificial intelligence model. This parameter is used to represent the complete two-dimensional spatial region of the character in the initial text image. The region intersection-union ratio refers to the ratio of the intersection area to the union area of ​​any two regions. This ratio is used to quantify the degree of spatial overlap between the two regions.

[0074] In one embodiment, the region intersection-union ratio can be calculated as follows: for each pair of first character regions and second character regions, calculate the pixel area of ​​their intersection and divide it by the union of the pixel areas of the two regions to obtain the corresponding region intersection-union ratio.

[0075] In one embodiment, to ensure calculation accuracy, the region coordinates can be integerized or pixel-level aligned to reduce calculation errors and improve the consistency of the cross-union ratio evaluation.

[0076] S11032. The second character region whose intersection-union ratio with the first character region is greater than the intersection-union ratio threshold is determined as the character position of the character object corresponding to the first character region.

[0077] The first character region refers to the point-level spatial parameters of the character object in the initial text image obtained after recognizing the character content through the first artificial intelligence model. These point-level spatial parameters can be used to represent the horizontal position of the character in the initial text image. The second character region refers to the region-level spatial parameters of the character object in the initial text image detected by the second artificial intelligence model. These parameters represent the complete two-dimensional spatial region of the character in the initial text image. The intersection-union ratio (IUGR) is the ratio of the intersection area to the union area of ​​any two regions, used to measure the degree of spatial overlap between the two regions. The IUGR threshold is a preset value used to filter out sufficiently overlapping second character regions to match the corresponding first character regions. The character position refers to the final determined two-dimensional coordinate parameters of the character object in the initial text image, used for subsequent character cropping, reconstruction, and text reconstruction processing. In determining the character position, since the low-precision character region of the first character region corresponds to character content information, while the high-precision character region of the second character region does not, i.e., the first character region indicates the approximate position of a character, while the second character region indicates a precise position containing a character, the precise position of a character object can be determined by calculating the IUGR of the first and second character regions. In one embodiment, the character position can be determined by: calculating the intersection-union ratio (IUU) of each second character region with the corresponding first character region, taking the second character region with an IUU greater than a threshold as a valid match, and using its coordinate information as the character position of the character object.

[0078] In one embodiment, the character position can be determined by: sequentially traversing the first character region, and for each traversed first character region, performing an intersection-union (IU) ratio (OCR) with each second character region. If the IU result is greater than 0.5, the second character region is used instead of the coarsely located position of the first character region as the character position. This makes the predicted character position more accurate. For example, suppose there are character objects A and B, where the first character region corresponding to character object A is a1, and the second character region corresponding to character object A is a2, while the first character region corresponding to character object B is b1, and the second character region corresponding to character object B is b2. Since the computer cannot directly map the two input character objects to each other during the detection process, it is necessary to calculate the IU for each character position to determine the correspondence between the two output character objects. For character object A, we need to calculate the intersection-union ratio (IUR) of its first character region a1 with both its second character region a2 and the second character region b2 of character object B. Since the first character region a1 and the second character region a2 correspond to the same character object, their IUR will exceed 0.5. This process allows us to establish the correspondence between the character objects output by the two models. Similarly, for the first character region b1 of character object B, we need to calculate its IUR with both its second character regions a2 and b2 to determine the correspondence.

[0079] In one embodiment, the obtained character positions can be used as input for subsequent character cropping, reconstruction, or text combination modules to ensure the consistency and reliability of character-level information with the image spatial layout.

[0080] S120. Based on the character position, crop the character object from the initial text image to obtain the initial character image corresponding to the character object.

[0081] In some embodiments, the initial character image refers to local image data extracted from the initial text image through a cropping operation, containing only a single character object pixel region, for subsequent character reconstruction, enhancement, or stylization processing.

[0082] In one embodiment, the cropping operation can be performed by: using the bounding box coordinates of the character position as a reference, cropping the corresponding rectangular image block in the initial text image, and outputting the image block as the initial character image.

[0083] In one embodiment, to improve the reliability of the cropping result, the method of determining the bounding box may include: adding fixed or adaptive extended margins based on the character object positioning result to compensate for the character stroke extension area or noise reduction requirements.

[0084] In one embodiment, the cropped initial character image can be further preprocessed by size normalization or pixel alignment to adapt to the input specifications of the subsequent reconstruction model.

[0085] S130. Reconstruct the character image by using the character content and its corresponding initial character image to obtain the reconstructed character image corresponding to the initial character image.

[0086] In some embodiments, character content refers to the text content corresponding to the character object at the semantic level, which is used to constrain the reconstruction model to generate character structures that conform to semantics; the initial character image refers to local image data containing the original visual form of the character object obtained based on the cropping step, which is used to provide visual references such as stroke texture, contour structure and local pixel distribution; the reconstructed text image refers to the target text image generated through reconstruction processing that is enhanced in terms of character clarity, structural integrity or style consistency.

[0087] In one embodiment, the reconstruction process can be performed by using character content as semantic input, combining it with an initial character image as visual input, and generating the corresponding character reconstruction result through a character reconstruction network, an image enhancement model, or an attention-based generative model.

[0088] In one embodiment, the reconstruction process can be: super-resolution processing, noise reduction reconstruction processing, deblurring reconstruction processing, image enhancement reconstruction processing, deep learning reconstruction processing, or interpolation-based reconstruction processing.

[0089] In one embodiment, the method for stitching the reconstructed text image can be: according to the spatial order of the character positions, the reconstruction results of each character are mapped back to their corresponding two-dimensional layout areas, and a complete reconstructed text image is constructed through a fusion algorithm.

[0090] Optionally, Figure 6 A flowchart of a text image reconstruction method provided in an embodiment of this application is given. (Reference) Figure 6 The text image reconstruction method specifically includes: S1301. Determine the prior features corresponding to the character object based on the initial character image and character content.

[0091] Among them, the initial character image refers to the local image data containing only a single character object obtained by cropping the initial text image; the character content refers to the specific text content corresponding to the character object at the semantic level, which is used to constrain the subsequent feature extraction and reconstruction process; the prior features refer to the comprehensive feature information describing the structure, texture, stroke shape, spatial distribution or semantic constraints of the character object, which can be used to guide character reconstruction or enhancement processing.

[0092] In one embodiment, the prior features can be determined by: extracting features from the initial character image to obtain edges, textures, stroke distributions, or local pixel patterns, and combining this with the semantic information of the character content to generate a prior feature vector of the character object, and then determining the generated prior feature vector as the prior features.

[0093] In one embodiment, prior features may include structural features, visual features, and semantic features, wherein structural features refer to the geometric shape of character strokes, visual features refer to image texture or pixel distribution patterns, and semantic features refer to the category or contextual constraint information of character content.

[0094] In one embodiment, the obtained prior features can be used as input to the character reconstruction model to guide the achievement of stroke continuity, morphological integrity, and character content consistency during the reconstruction process.

[0095] Optionally, Figure 7 A flowchart of a priori feature generation method provided in an embodiment of this application is given. (Reference) Figure 7 The prior feature generation method specifically includes: S13011. Obtain the character encoding corresponding to the character content.

[0096] Character content refers to the specific text content corresponding to a character object at the semantic level; character encoding refers to the standardized encoding value used to uniquely identify the character content, such as Unicode encoding, ASCII encoding, or custom character mapping values, which are used to achieve a unified representation and subsequent processing of characters in a computer system.

[0097] In one embodiment, the character encoding can be obtained by searching for the encoding value corresponding to the character content in a predefined character dictionary or encoding table, and using the found encoding value as the character encoding. Character encoding can serve as a semantic identifier for character objects.

[0098] In one embodiment, character encoding can be used for generating character feature vectors, semantic constraints on character reconstruction model inputs, or alignment processing of character content with image features.

[0099] In one embodiment, to ensure the consistency of character encoding, characters from different sources or languages ​​can be uniformly mapped, thereby standardizing the representation of character content during reconstruction or correction.

[0100] S13012. Identify the character style corresponding to the character object based on the initial character image.

[0101] The initial character image refers to the local image data containing only a single character object obtained by cropping the initial text image; the character object refers to the text unit that can be independently located in the initial text image; the character style refers to the visual characteristics of the character object, including stroke thickness, stroke shape, tilt angle, decorative elements and overall writing style, which are used to guide character reconstruction and enhancement processing.

[0102] In one embodiment, character style identification can be achieved by inputting an initial character image into a deep learning model or a style recognition model, extracting texture features, analyzing strokes, or encoding convolutional features from the initial character image, classifying or regressing the visual features of the characters, and outputting the corresponding character style.

[0103] In one embodiment, character style may include structural style, writing style, and decorative style, wherein structural style refers to stroke proportions and layout, writing style refers to slant and stroke continuity, and decorative style refers to additional visual embellishments or special effects elements.

[0104] In one embodiment, the identified character style can be used as input features for the character reconstruction model to ensure that the reconstructed characters are visually consistent with the style of the original character object, while improving the naturalness and aesthetics of the reconstructed text image.

[0105] S13013. Input the character encoding and its corresponding character style into the trained fourth artificial intelligence model to obtain the prior features corresponding to the character object.

[0106] Among them, character encoding refers to the standardized encoding value used to uniquely identify character content, such as Unicode encoding or custom character mapping values; character style refers to the visual performance characteristics of character objects, including stroke thickness, stroke shape, tilt angle, and decorative elements; the fourth artificial intelligence model refers to a deep learning model trained to generate prior character features, such as a model based on multimodal fusion networks, image encoders, and semantic encoders; prior features refer to feature information that comprehensively describes the structure, stroke texture, visual style, and semantic constraints of character objects, and is used to guide character reconstruction or enhancement processing.

[0107] In one embodiment, the prior features can be generated by fusing the semantic vector of the character encoding and the visual feature vector of the character style through a fourth artificial intelligence model, and outputting a multi-dimensional prior feature representation of the corresponding character object to obtain the prior features.

[0108] In one embodiment, prior features may include structural features, visual features, and semantic features, wherein structural features refer to the geometric shape of character strokes, visual features refer to image textures or stroke patterns, and semantic features refer to the semantic constraint information provided by character encoding.

[0109] In one embodiment, the obtained prior features can be used as input to the subsequent character reconstruction model to ensure that the reconstructed characters are consistent with the original character objects in terms of stroke shape, style consistency, and semantic accuracy.

[0110] S1302. Determine the corresponding initial image feature map based on the initial text image.

[0111] The initial text image refers to text image data containing the original character distribution and visual texture information; the initial image feature map refers to the multi-scale, multi-channel visual feature representation extracted from the initial text image, used to describe the structural information, texture distribution and local pixel patterns of the image, in order to assist in the reconstruction and enhancement of character objects.

[0112] In one embodiment, the initial image feature map can be determined by: inputting the initial text image into a trained image feature extraction model, obtaining feature maps in the model's intermediate or output layers, and using these obtained feature maps as the initial image feature map. The initial image feature map is then used for subsequent character reconstruction or image enhancement processing.

[0113] In one embodiment, the initial image feature map may include low-level texture features, high-level structural features, and multi-scale contextual information, wherein the low-level features describe character edges and stroke textures, the high-level features describe character structure and layout, and multi-scale feature fusion can enhance the perception of the overall shape of the character.

[0114] In one embodiment, the obtained initial image feature map can be used as input to a character reconstruction model or enhancement model to improve the reconstructed text image in terms of stroke integrity, style consistency, and visual naturalness.

[0115] Optionally, Figure 8 A flowchart of an initial image feature map generation method provided in an embodiment of this application is given. (Reference) Figure 8 The initial image feature map generation method specifically includes: S13021. Input the initial text image into the fifth artificial intelligence model that has been trained.

[0116] The initial text image refers to text image data containing the original character distribution and visual texture information; the fifth artificial intelligence model can be a U-shaped neural network model or other deep convolutional neural network models with encoder-decoder structure, which fuses low-level and high-level features through skip connections, and is used for tasks such as pixel-level feature extraction, reconstruction or enhancement of input images.

[0117] In one embodiment, the initial text image can be input by normalizing its size, standardizing its channels, and performing necessary noise suppression before using it as model input, so as to ensure that the model can accurately extract the spatial structure and texture information of the characters during the inference phase.

[0118] In one embodiment, the fifth artificial intelligence model can generate multi-scale feature maps, extract local and global information through an encoder, and reconstruct character structures through a decoder, thereby achieving character stroke enhancement, edge repair, and image detail preservation.

[0119] In one embodiment, the feature representation obtained by inputting the initial text image into the fifth artificial intelligence model can be used for subsequent character object reconstruction, texture refinement, or visual enhancement processing to improve the clarity and visual consistency of the reconstructed text image.

[0120] S13022. Based on the coding path of the fifth artificial intelligence model, extract the semantic features of the initial text image at various scales.

[0121] Among them, the fifth artificial intelligence model refers to a deep convolutional neural network with an encoder-decoder structure and skip connections; the encoding path refers to the part of the fifth artificial intelligence model responsible for layer-by-layer downsampling and feature extraction, which is used to capture local and global information of the image; the image semantic features refer to feature maps that represent character structure, stroke distribution, edge information and local texture at different spatial scales, which are used for subsequent character reconstruction and enhancement processing.

[0122] In one embodiment, the method for extracting image semantic features can be as follows: Multiple convolutional and pooling operations are performed on the initial text image through an encoding path, generating feature maps at corresponding scales in each layer. These generated feature maps are then used as image semantic features. Image semantic features can represent the multi-scale semantics of the initial text image.

[0123] In one embodiment, the image semantic features at various scales may include low-level features and high-level features, wherein the low-level features describe character stroke texture and edges, and the high-level features describe character structural layout and global morphology.

[0124] In one embodiment, the extracted multi-scale image semantic features can be combined with skip connection paths for feature fusion in the decoder stage, thereby guiding pixel-level reconstruction and visual enhancement of character objects and improving the clarity and naturalness of the reconstructed text images.

[0125] S13023. The image semantic features with the smallest scale are used as the initial input of the decoding path of the fifth artificial intelligence model. The features output by the previous scale of the decoding path are upsampled step by step to obtain the upsampled features.

[0126] Among them, the fifth artificial intelligence model refers to a deep convolutional neural network with an encoder-decoder structure and skip connections; the smallest scale image semantic feature refers to the feature map with the lowest resolution but the richest semantic information obtained by downsampling the last layer of the encoding path; the decoding path refers to the part of the fifth artificial intelligence model that upsamples and fuses features layer by layer, which is used to restore the spatial resolution and detail information of the input image; the upsampled feature refers to the feature representation obtained after each layer of the decoding path gradually restores the resolution, which is used to guide the pixel-level reconstruction and visual enhancement of character objects.

[0127] In one embodiment, the stepwise upsampling method can be: restoring the feature map of the previous scale to the spatial resolution of the previous coding scale through methods such as transposed convolution, bilinear interpolation, or upsampling convolution.

[0128] In one embodiment, the obtained upsampled features can be fused with the image semantic features at the scale corresponding to the encoding path through skip connections to enhance the ability of the decoding stage to perceive character structure and stroke details.

[0129] In one embodiment, the upsampling features can be used as input to subsequent character reconstruction, texture refinement, or image enhancement modules to generate clear, stylistically consistent, and visually natural reconstructed text images.

[0130] S13024. By fusing image semantic features and upsampling features through skip connections, the initial image feature map corresponding to the initial text image is obtained.

[0131] Among them, image semantic features refer to the multi-layer features extracted at each scale of the encoding path, which are used to describe character structure, stroke distribution and local texture information; upsampling features refer to the feature representations obtained by upsampling at each level of the decoding path, which are used to restore the spatial resolution and detail information of the image; skip connections refer to the mechanism in the fifth artificial intelligence model that connects or fuses the features of the encoding path with the upsampling features of the decoding path at the corresponding scale to enhance the detail expression ability of the features; the initial image feature map refers to the multi-scale, pixel-level feature representation after fusion, which is used to guide character object reconstruction and visual enhancement processing.

[0132] In one embodiment, skip connection fusion can be achieved by concatenating or weighting the corresponding scale image semantic features and upsampled features along the channel dimension, and then further fusing them through convolution operations to generate an enhanced feature map.

[0133] In one embodiment, the initial image feature map can simultaneously contain low-level texture information and high-level structural information, thereby preserving both the details of character strokes and the semantic consistency of the overall character shape.

[0134] In one embodiment, the obtained initial image feature map can be used as input to a subsequent character reconstruction module or enhancement module to generate visually clear, stylistically consistent, and semantically accurate reconstructed text images.

[0135] S1303. Determine the position code corresponding to the character object based on the character position.

[0136] Among them, character position refers to the two-dimensional spatial coordinates of the character object in the initial text image obtained by fusing the first character region and the second character region, such as the bounding box, center point or baseline position; position encoding refers to converting the character position information into a numerical or vector representation that the model can process, which is used to provide spatial constraints in the character reconstruction or enhancement process; character object refers to the independently identifiable character unit in the initial text image, which is used for subsequent character cropping, reconstruction and text combination processing.

[0137] In one embodiment, the positional encoding can be determined by mapping the two-dimensional coordinates of the character position to a normalized spatial vector, and then using the resulting spatial vector as the positional encoding. The positional encoding is used to represent the spatial distribution of character objects in an image.

[0138] In one embodiment, positional encoding can combine character region size, aspect ratio, and relative position to enhance the reconstruction model's ability to perceive the position and layout of character strokes.

[0139] In one embodiment, the obtained positional encoding can be used as input features for a character reconstruction model to guide the accurate reconstruction and arrangement of character objects in a pixel-level image, ensuring that the spatial structure of the reconstructed text image is consistent with the initial text image.

[0140] Optionally, the character position is input into the trained sixth artificial intelligence model, which maps the character position to the position code corresponding to the character object.

[0141] Among them, character position refers to the two-dimensional spatial coordinates of the character object in the initial text image obtained by fusing the first character region and the second character region, such as the bounding box, center point, or baseline position; the sixth artificial intelligence model refers to a deep learning model that has been trained to map two-dimensional spatial coordinates into numerical vectors or feature representations that can be used for neural network processing. The sixth artificial intelligence model can be a position encoding model; position encoding refers to the spatial vector representation of the character object in the image output by the model, which is used to provide precise spatial constraints in the process of character reconstruction, enhancement, and arrangement.

[0142] In one embodiment, the method of mapping character positions to position codes can be: normalizing the two-dimensional coordinates through a sixth artificial intelligence model and then performing linear or nonlinear transformations to generate feature vectors or position heatmaps, so as to preserve the relative position and spatial layout information of character objects.

[0143] In one embodiment, the obtained positional encoding can be combined with character prior features and initial image feature maps as input to a character reconstruction model to guide pixel-level reconstruction of character objects and spatial layout consistency of reconstructed text images.

[0144] In one embodiment, to improve the accuracy of positional encoding, character size, aspect ratio, and the relative positions of adjacent characters can be combined when training the sixth artificial intelligence model, so that the positional encoding can more accurately reflect the spatial structure of the character object in the image.

[0145] S1304. Input the initial image feature map, position encoding and prior features into the trained third artificial intelligence model to obtain the reconstructed text image corresponding to the initial text image.

[0146] Among them, the initial image feature map refers to the multi-scale, pixel-level feature representation extracted by the fifth artificial intelligence model, which is used to describe the structure, texture and local details of the initial text image; the positional encoding refers to the numerical vector or feature map obtained by mapping the character position, which is used to provide spatial constraints for the character object; the prior features refer to the comprehensive feature information generated by the character content, character encoding and character style, which is used to guide the reconstruction of the shape, stroke and visual style of the character object; the third artificial intelligence model refers to the deep learning model trained to achieve super-resolution reconstruction of text images, such as models based on multimodal feature fusion, attention mechanism or convolutional-residual network structure; the reconstructed text image refers to the reconstruction result that is superior to the initial text image in terms of spatial resolution, detail clarity and visual naturalness.

[0147] In one embodiment, the method for generating a reconstructed text image can be as follows: the initial image feature map, position encoding, and prior features are input into a third artificial intelligence model through a feature fusion module. The model refines the character strokes and textures through multi-layer convolution, residual connections, and attention mechanisms to generate a high-resolution text image, and the generated high-resolution text image is used as the reconstructed text image.

[0148] In one embodiment, reconstructing a text image can simultaneously maintain the accuracy of character content, the integrity of strokes, stylistic consistency, and spatial layout consistency, thereby improving the readability and visual quality of the text image.

[0149] In one embodiment, the resulting reconstructed text image can be used as input for subsequent text analysis, visual presentation, or editing operations, providing high-quality image data for text recognition, typography, and enhancement.

[0150] Optionally, Figure 9 A schematic diagram of a text image processing device according to an embodiment of this application is provided. (Reference) Figure 9The text image processing device specifically includes: a text recognition and positioning module 11, a text prior generation module 12, and a text super-resolution module 13.

[0151] For example, the character recognition and localization module 11 is responsible for accurately extracting character content from the initial text image and determining the character position of each character object. The output of the character recognition and localization module 11 includes the category information of each character object and its corresponding bounding box, providing basic data for the generation of prior features and super-resolution processing of subsequent character objects. The character recognition and coarse localization algorithm, the NLP character correction algorithm, and the YOLO-based character detection algorithm together constitute the character recognition and localization module 11. After multi-stage processing by this module, the text image can obtain high-precision character recognition results and precise position coordinates of each character, ensuring a strict one-to-one correspondence between character content and its spatial position. In the coarse recognition and initial localization stage, a lightweight character recognition network is used to perform fast character content recognition on the initial text image, and the first character position at the character level is generated through the preliminary localization algorithm. This stage outputs the character content and the first character position, which are used for constraints and calibration of subsequent detection tasks. In the NLP text correction stage, semantic-level error correction is performed based on character content using language models or syntactic analysis models. This includes correcting misidentification of character shape similarity, reconstructing contextual consistency, and restoring natural language fluency, outputting high-confidence character content with semantic correction. In the YOLO-based fine localization stage, the YOLO detection network performs high-precision character detection on the initial text image, generating second character positions. These second character positions are then aligned with the NLP-corrected character content to achieve a precise mapping between character content and pixel spatial location.

[0152] In one embodiment, the text prior generation module 12 generates a prior image corresponding to prior features consistent with the initial character image and blurred features, based on the character content and character position. This prior image can reflect the character shape and local texture features, and is used as guiding information for the super-resolution module to improve the accuracy and visual consistency of text restoration.

[0153] In one embodiment, the text super-resolution module 13 takes the feature map of the initial text image, the prior image, and the positional encoding of each character as input, and performs super-resolution processing through a convolutional neural network to obtain a high-resolution text image. The text super-resolution module 13 fuses visual texture information, prior information, and positional constraints to achieve fine-grained character-level restoration. The text super-resolution module 13 is used to restore low-resolution text images to high-resolution, clear images. This module achieves fine-grained reconstruction with character-level constraints by fusing blurred image features, character positional information, and character prior features generated based on StyleGAN (Style-based Generative Adversarial Network).

[0154] Optionally, Figure 10 An algorithmic framework diagram of a text image processing method provided in an embodiment of this application is given. (Reference) Figure 10 The text image processing method specifically includes: S201, CRNN and post-processing are used for text recognition and coarse character localization, and NLP correction is performed.

[0155] For example, for an initial input text image, it is fed into a CRNN (Convolutional Recurrent Neural Network) model, and the output of the CRNN model is post-processed to achieve text recognition and rough character localization of the initial text image, obtaining the character content and its corresponding first character position. First, the height of the initial text image is scaled to 32 pixels, and the width is scaled proportionally to the height. The scaled initial text image is then input into the CRNN model for time-step prediction, obtaining the character probability distribution sequence for each time step. Subsequently, the character probability distribution sequence is decoded, and a greedy decoding algorithm can be used in the decoding process. If the probability of a certain character remains the highest for a continuous period of time steps, then that character is considered to appear in the corresponding image region. Since the time steps... The corresponding feature map of the first Column, and the first Each column corresponds to a receptive field of the initial text image, thus allowing for an approximate reconstruction of the horizontal position of the character object: Character position ≈ Time step index range × Downsampling ratio. And the time step... The corresponding image region is:

[0156]

[0157] in, The original image width, The length of the feature sequence. For CNN's stride, This represents the range of time step indexes.

[0158] In one embodiment, NLP (Natural Language Processing) correction is performed on the decoded character content, and semantic-level text repair processing is executed when errors exist in the character recognition result. The NLP algorithm can perform "replacement correction" on the recognized character sequence based on contextual semantic relationships, without adding or deleting characters. That is, it only performs one-to-one character replacement operations within the existing character set to improve the semantic accuracy of the recognized text. In this mode, the character length remains unchanged, and the text structure remains stable, effectively avoiding character position mapping deviations caused by character additions or deletions. For example, the initial recognition result is "There is a large elephant on the African savanna," while the NLP correction result is "There is an elephant on the African savanna." This correction mechanism can correct erroneous characters while maintaining a consistent number of characters, providing more reliable character content input for subsequent text prior generation and text super-resolution modules.

[0159] S202, Text detection based on YOLO.

[0160] For example, the initial text image is input into a text detection model based on YOLO (You Only Look Once, a real-time object detection algorithm) to obtain the second character position of the character objects in the initial text image. The YOLO-based text detection algorithm uses a trained character detection model to perform character-level object detection on the input text image. Based on the YOLO framework, this model, trained on a large number of character samples, can simultaneously output the character category confidence and its corresponding spatial bounding box in a single forward inference. In actual processing, the model can perform pixel-by-pixel perception and feature regression on the text region, thereby locating the precise position of each character in the image. The detection results are presented as rectangular boxes representing the spatial range of the characters in the image; the area marked with a red rectangle is the model's detection output. Each detection box establishes a one-to-one mapping relationship with the corresponding character content, providing accurate spatial constraint information for subsequent text prior generation and text super-resolution modules. This detection process features strong real-time performance, high positioning accuracy, and good adaptability to character deformation and lighting changes, ensuring stable character position output even in complex scenes.

[0161] In one embodiment, to improve the accuracy of character localization, after character recognition is completed, the first character region provided by the CRNN is calibrated based on the detection results. Specifically, each coarse localization box output by the CRNN, i.e., the first character region, is sequentially traversed, and the Intersection over Union (IoU) is calculated with each character box output by the YOLO character detection model, i.e., the second character region. When the IoU between a detection box and the CRNN coarse localization box is greater than 0.5, the position of the detection box replaces the coarse localization position provided by the CRNN. ​​Through this matching and replacement strategy, the localization deviation generated by the CRNN under character compression, deformation, or background interference can be effectively corrected. The formula for calculating IoU is as follows:

[0162] in, The rough bounding box output by the CRNN, i.e., the first character region; This is the bounding box for the YOLO detection algorithm, specifically the second character region.

[0163] The calibrated character positions more accurately reflect the actual spatial distribution of characters in the image and maintain a one-to-one correspondence with the character content. The green line represents the character center position, visually demonstrating the improved accuracy after position correction. This position fusion strategy combines the content recognition capabilities of CRNN with the precise localization capabilities of YOLO, significantly enhancing both the accuracy and stability of the final character position output. This provides reliable spatial constraint information for subsequent text prior generation and super-resolution modules.

[0164] After the above steps, the accurate character content and corresponding character position can be obtained, as shown below. The character content sequence is [character1, character2, ..., charactern], and the character position sequence is [position1, position2, ..., positionn].

[0165] S203. Each character object is mapped to a character style feature vector W.

[0166] For example, based on the obtained character positions, multiple initial character images are cropped from the initial text image, and a character style feature vector W is obtained for each character object using these initial text images. Specifically, based on the obtained character positions, the corresponding character regions in the initial text images are cropped out, and these character regions are input into a style prediction network, outputting a 512-dimensional character style feature vector W. The character style feature vector W describes the overall style attributes of the text in the image, such as font type, stroke thickness, slant, edge sharpness, and other style factors. The character style feature vector W is used to control the final visual representation of the characters in subsequent generation processes.

[0167] S204, StyleGAN.

[0168] For example, the character encoding and its corresponding character style feature vector W are input into StyleGAN (Style-based Generative Adversarial Network) to obtain prior features. Specifically, the character content of each identified character object is converted into a character encoding, and a corresponding "initial template" is obtained for each character object by querying a pre-trained character template library. The character template contains the basic stroke structure and standard morphological information of the character object, which is equivalent to the structured baseline morphology of the character. The StyleGAN network is used to combine the character encoding and the character style feature vector W to generate the final prior features. First, based on the character encoding, the initial template of the corresponding character is retrieved from the template library to ensure that the generated target character will not have character content errors. The character style feature vector W is input into the multi-scale modulation structure of StyleGAN to perform stylization transformation on the character template. Style modulation affects the thickness, contour sharpness, texture features, etc. of the character, so that the character gradually fits the overall style of the blurred image. Each layer of StyleGAN performs style control based on the character style feature vector W, while gradually enlarging and refining the character features at different resolutions. The final output prior feature map is clear and has a consistent style, which can be used to guide subsequent super-resolution reconstruction.

[0169] S205 and UNet feature extraction.

[0170] For example, the initial text image is input into a UNet (U-shaped Convolutional Network) for feature extraction to obtain an initial image feature map. Specifically, the low-resolution initial text image is input into the UNet network to extract multi-scale features, including edge features, texture features, and overall structural features. The UNet's encoder-decoder structure can enhance detail representation while preserving global information, providing basic visual features for the super-resolution process.

[0171] S206, CNN super-resolution module.

[0172] For example, the initial image feature map, positional encoding, and prior features are input into the CNN super-resolution module to obtain the reconstructed text image corresponding to the initial text image. Specifically, based on the character position obtained by the character recognition and localization module 11, a positional encoding is generated for each character object, and this encoding is mapped into the feature space. The positional encoding guides the network to focus on the region where the character is located, enabling the network to focus on enhancing the character region during the reconstruction process, thereby improving the reconstruction quality of character edges, strokes, and local structures. The prior features generated by StyleGAN based on the character template and character style feature vector W contain the shape and style information that the character should have. The prior features provide the super-resolution module with a "high-confidence target shape," which can provide an effective supplement when the network has difficulty recovering stroke details from the blurred original image. After spatial alignment and channel-level encoding of the positional encoding, prior features, and initial image feature map through the feature fusion module, they are jointly input into the CNN super-resolution module. Based on the fused multimodal features, the CNN super-resolution module performs convolutional inference and layer-by-layer reconstruction, and finally outputs a high-resolution reconstructed image. While maintaining the original layout, the image significantly improves the clarity of character edges, the recognizability of strokes, and the overall visual quality.

[0173] Based on the above embodiments, Figure 11 This is a structural block diagram of a text image processing apparatus provided in an embodiment of this application. (Reference) Figure 11 The text image processing device provided in this embodiment specifically includes: a feature extraction module 21, a character cropping module 22, and a super-resolution reconstruction module 23.

[0174] The feature extraction module 21 is configured to acquire the initial text image to be reconstructed and determine the character content and character position corresponding to the character object in the initial text image; the character cropping module 22 is configured to crop the character object from the initial text image according to the character position to obtain the initial character image corresponding to the character object; and the super-resolution reconstruction module 23 is configured to perform reconstruction processing using the character content and its corresponding initial character image to obtain the reconstructed text image corresponding to the initial text image.

[0175] Based on the above embodiments, the number of character objects is at least one, and the feature extraction module 21 includes: a first model unit configured to input the initial text image into a trained first artificial intelligence model to obtain the character content corresponding to at least one character object in the initial text image and the first character region in which it is located; a second model unit configured to input the initial text image into a trained second artificial intelligence model to obtain the second character region corresponding to at least one character object in the initial text image; and a position determination unit configured to determine the character position corresponding to each character object based on at least one first character region and at least one second character region.

[0176] Based on the above embodiments, the first model unit includes: a first model subunit configured to input an initial text image into a trained first artificial intelligence model to obtain a character probability distribution sequence arranged by time steps; a sequence decoding subunit configured to decode the character probability distribution sequence to obtain the character content and time step index range corresponding to at least one character object in the initial text image; and a region mapping subunit configured to map the time step index range to a horizontal region of the initial text image to determine the first character region where at least one character object corresponding to the time step index range is located.

[0177] Based on the above embodiments, the first model unit further includes: a character combination subunit, configured to combine character content to obtain text content corresponding to the initial text image; a text correction subunit, configured to perform text correction processing on the text content to obtain corrected content corresponding to the text content; and a character correction subunit, configured to correct the character content according to the corrected content to obtain corrected character content.

[0178] Based on the above embodiments, the position determination unit includes: an intersection-union ratio calculation subunit, configured to calculate the intersection-union ratio of each pair of the first character regions and each second character regions; and a position determination subunit, configured to determine the second character regions whose intersection-union ratio with the first character regions is greater than the intersection-union ratio threshold as the character positions of the character objects corresponding to the first character regions.

[0179] Based on the above embodiments, the super-resolution reconstruction module 23 includes: a prior feature unit, configured to determine the prior features corresponding to the character object based on the initial character image and character content; an image feature unit, configured to determine the corresponding initial image feature map based on the initial text image; a position encoding unit, configured to determine the position encoding corresponding to the character object based on the character position; and a third model unit, configured to input the initial image feature map, position encoding, and prior features into a trained third artificial intelligence model to obtain the reconstructed text image corresponding to the initial text image.

[0180] Based on the above embodiments, the prior feature unit includes: a character encoding subunit, configured to acquire the character encoding corresponding to the character content; a character style subunit, configured to identify the character style corresponding to the character object based on the initial character image; and a fourth model subunit, configured to input the character encoding and its corresponding character style into the trained fourth artificial intelligence model to obtain the prior features corresponding to the character object.

[0181] Based on the above embodiments, the image feature unit includes: a fifth model subunit, configured to input the initial text image into the trained fifth artificial intelligence model; a semantic feature subunit, configured to extract image semantic features of the initial text image at various scales based on the encoding path of the fifth artificial intelligence model; an upsampling subunit, configured to use the smallest scale image semantic feature as the initial input to the decoding path of the fifth artificial intelligence model, and upsample the features output at the previous scale of the decoding path step by step to obtain upsampled features; and a skip connection subunit, configured to fuse the image semantic features and upsampled features through skip connections to obtain the initial image feature map corresponding to the initial text image.

[0182] Based on the above embodiments, the position encoding unit includes: an encoding model subunit, configured to input the character position into the trained sixth artificial intelligence model, and to map the character position to the position encoding corresponding to the character object through the sixth artificial intelligence model.

[0183] The text image processing apparatus provided in this application embodiment, by constructing a hierarchical collaborative processing system composed of a feature extraction module 21, a character cropping module 22, and a super-resolution reconstruction module 23, achieves character content parsing, character region extraction, and high-precision reconstruction of text images, improving the clarity and readability of blurred text images while ensuring the semantic correctness of characters. The feature extraction module 21 possesses text image parsing and character-level information extraction capabilities, enabling it to acquire the initial text image to be reconstructed and determine the character content and spatial position of each character object in the image through a recognition algorithm. In this stage, the initial image is structured into a character sequence and a position sequence, providing an accurate semantic basis and spatial constraints for subsequent character-level reconstruction. The character cropping module 22 undertakes the character region extraction task, cropping the corresponding character objects from the initial text image based on the aforementioned position data to generate an initial character image for each character. This module can ensure the matching of the cropped region and the character position while preserving the local details of the character, thereby enabling the subsequent reconstruction network to obtain a clear local input region, ensuring the targetness and effectiveness of the reconstruction processing. The super-resolution reconstruction module 23 is responsible for performing image reconstruction processing based on the character content and the corresponding initial character image. This module jointly models character semantic information and character image information, generates high-resolution character regions through a super-resolution network, and integrates them into a complete reconstructed text image based on character-level reconstruction. The final output reconstructed image shows significant improvements in stroke detail, character edge sharpness, and overall visual quality, providing high-quality input for applications such as text recognition, image enhancement, and scene text restoration. Through the coordinated operation of character content extraction, character region separation, and super-resolution generation, this device maintains stable reconstruction capabilities under various degradation conditions such as character blurring, noise interference, and complex backgrounds, improving the restoration effect and application reliability of text images. It is suitable for enhancing low-resolution scanned documents, blurry photographed text, and various low-quality text images.

[0184] The text image processing apparatus provided in this application embodiment can be used to execute the text image processing method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0185] Figure 12 This is a schematic diagram of the structure of a text image processing device provided in an embodiment of this application, with reference to... Figure 12 The text image processing device includes a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 and the number of memories 32 in the text image processing device can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of the text image processing device can be connected via a bus or other means.

[0186] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the text image processing method in any embodiment of this application (e.g., data acquisition module 11, flow rate determination module 12, and liquid output determination module 13 in the text image processing device). The memory 32 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory 32 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0187] The communication device 33 is used for data transmission.

[0188] The processor 31 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 32, thereby realizing the above-mentioned text and image processing method.

[0189] Input device 34 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 35 may include display devices such as a display screen.

[0190] The text image processing device provided above can be used to execute the text image processing method provided in the above embodiments, and has corresponding functions and beneficial effects.

[0191] This application also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a text image processing method. The text image processing method includes: acquiring an initial text image to be reconstructed; determining the character content and character position corresponding to a character object in the initial text image; cropping the character object from the initial text image according to the character position to obtain an initial character image corresponding to the character object; and performing reconstruction processing using the character content and its corresponding initial character image to obtain a reconstructed text image corresponding to the initial text image.

[0192] Storage medium—any type of memory device or storage device. The term "storage medium" is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which a program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0193] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the text image processing method described above, but can also perform related operations in the text image processing method provided in any embodiment of this application.

[0194] The text image processing apparatus, storage medium, and text image processing device provided in the above embodiments can execute the text image processing method provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the text image processing method provided in any embodiment of this application.

[0195] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application. The scope of this application is determined by the scope of the claims.

Claims

1. A text image processing method, characterized in that, include: Obtain the initial text image to be reconstructed, and determine the character content and character position corresponding to the character objects in the initial text image; Based on the character position, the character object is cropped from the initial text image to obtain the initial character image corresponding to the character object; The reconstructed text image is obtained by using the character content and its corresponding initial character image for reconstruction processing.

2. The text image processing method according to claim 1, characterized in that, The number of character objects is at least one, and determining the character content and character position corresponding to the character objects in the initial text image includes: The initial text image is input into the first trained artificial intelligence model to obtain the character content corresponding to at least one character object in the initial text image and the first character region in which it is located. The initial text image is input into the trained second artificial intelligence model to obtain the second character region corresponding to at least one character object in the initial text image; The character position corresponding to each character object is determined based on at least one first character region and at least one second character region.

3. The text image processing method according to claim 2, characterized in that, The step of inputting the initial text image into the trained first artificial intelligence model to obtain the character content corresponding to at least one character object in the initial text image and the first character region it belongs to includes: The initial text image is input into the first trained artificial intelligence model to obtain a character probability distribution sequence arranged by time step; Decoding the character probability distribution sequence yields the character content and time step index range corresponding to at least one character object in the initial text image; The time step index range is mapped to the horizontal region of the initial text image to determine the first character region where at least one character object corresponding to the time step index range is located.

4. The text image processing method according to claim 3, characterized in that, After decoding the character probability distribution sequence to obtain the character content and time step index range corresponding to at least one character object in the initial text image, the method further includes: The character content is combined to obtain the text content corresponding to the initial text image; The text content is subjected to text correction processing to obtain the corrected content corresponding to the text content; The character content is corrected according to the correction content to obtain the corrected character content.

5. The text image processing method according to claim 2, characterized in that, Determining the character position corresponding to each character object based on at least one first character region and at least one second character region includes: Calculate the region intersection-union ratio between each pairwise combination of the first character region and each second character region; The second character region whose intersection-union ratio with the first character region is greater than the intersection-union ratio threshold is determined as the character position of the character object corresponding to the first character region.

6. The text image processing method according to claim 1, characterized in that, The process of reconstructing the character content and its corresponding initial character image to obtain the reconstructed text image corresponding to the initial text image includes: Determine the prior features corresponding to the character object based on the initial character image and the character content; Determine the corresponding initial image feature map based on the initial text image; The position code corresponding to the character object is determined based on the character position; The initial image feature map, the position encoding, and the prior features are input into the trained third artificial intelligence model to obtain the reconstructed text image corresponding to the initial text image.

7. The text image processing method according to claim 6, characterized in that, The step of determining the prior features corresponding to the character object based on the initial character image and the character content includes: Obtain the character encoding corresponding to the character content; Identify the character style corresponding to the character object based on the initial character image; The character encoding and its corresponding character style are input into the trained fourth artificial intelligence model to obtain the prior features corresponding to the character object.

8. The text image processing method according to claim 6, characterized in that, The step of determining the corresponding initial image feature map based on the initial text image includes: The initial text image is input into the fifth artificial intelligence model that has been trained. Based on the encoding path of the fifth artificial intelligence model, the image semantic features of the initial text image at various scales are extracted; The image semantic features with the smallest scale are used as the initial input to the decoding path of the fifth artificial intelligence model. The features output by the previous scale of the decoding path are upsampled step by step to obtain the upsampled features. By fusing the image semantic features and the upsampled features through skip connections, an initial image feature map corresponding to the initial text image is obtained.

9. The text image processing method according to claim 6, characterized in that, Determining the position encoding corresponding to the character object based on the character position includes: The character position is input into the trained sixth artificial intelligence model, which then maps the character position to the position code corresponding to the character object.

10. A text image processing device, characterized in that, include: The feature extraction module is used to acquire the initial text image to be reconstructed and to determine the character content and character position corresponding to the character objects in the initial text image; A character cropping module is used to crop the character object from the initial text image according to the character position to obtain the initial character image corresponding to the character object; The super-resolution reconstruction module is used to perform reconstruction processing using the character content and its corresponding initial character image to obtain the reconstructed character image corresponding to the initial character image.

Images