Text-separation method and device, electronic equipment and storage medium

Abstract

The present application relates to the technical field of image processing, and provides a kind of graphic-text separation method, device, electronic equipment and storage medium.The method comprises: first, the pre-processing of image to be processed is carried out, and edge image, enhanced image and wavelet transform image are obtained, and according to wavelet transform image, the wavelet frequency domain feature corresponding to image to be processed is obtained;Then using the first network of graphic-text segmentation model, according to image to be processed, edge image, enhanced image and wavelet transform image, the multiscale feature corresponding to image to be processed is generated;And using the second network of graphic-text segmentation model, according to the wavelet frequency domain feature and multiscale feature corresponding to image to be processed, dynamic threshold is predicted and the probability graph of image to be processed is generated;Probability graph represents the probability that each pixel point in corresponding image belongs to text and non-text respectively;Finally, according to the probability graph of image to be processed, the graphic-text separation result of image to be processed is obtained.Thereby the accuracy and efficiency of graphic-text separation are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image-text separation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Image-text separation technology is an important research direction in the field of computer vision and image processing. Its core objective is to accurately extract text regions from complex images containing both text and non-text elements. With the acceleration of digitization, this technology plays a crucial role in scenarios such as document processing, invoice recognition, and archive digitization.

[0003] Traditional methods primarily achieve image-text separation by analyzing pixel grayscale distribution. However, this approach is prone to misidentifying background noise as text when faced with complex textured backgrounds, non-uniform lighting, or scenes with multiple fonts, thus affecting the separation results. Although there are currently deep learning-based methods for text detection to achieve image-text separation, they rely on fixed thresholds, resulting in poor performance when handling diverse text types. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and storage medium for separating images and text.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a method for separating text and images, the method comprising: The image to be processed is preprocessed to obtain an edge image, an enhanced image, and a wavelet transform image. Based on the wavelet transform image, the wavelet frequency domain features corresponding to the image to be processed are obtained. Using the first network of the image segmentation model, multi-scale features corresponding to the image to be processed are generated based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image. Using the second network of the image-text segmentation model, a dynamic threshold is predicted and a probability map of the image to be processed is generated based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively; Based on the probability map of the image to be processed, the image-text separation result of the image to be processed is obtained.

[0006] In an optional implementation, the image to be processed is preprocessed to obtain an edge image, an enhanced image, and a wavelet transform image. Based on the wavelet transform image, the wavelet frequency domain features corresponding to the image to be processed are obtained, including: Edge extraction is performed on the image to be processed to obtain the edge image; The image to be processed is enhanced to obtain the enhanced image; Perform wavelet transform on the image to be processed to obtain the wavelet transformed image; Feature extraction is performed on the wavelet transform image to obtain the wavelet frequency domain features corresponding to the image to be processed.

[0007] In an optional implementation, the first network includes a mapping layer and an extraction layer; Using the first network of the image segmentation model, multi-scale features corresponding to the image to be processed are generated based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image, including: Using the mapping layer, the dimensionality transformation is performed on the image to be processed, the edge image, the enhanced image, and the wavelet transform image to obtain the basic features corresponding to the image to be processed; Using the extraction layer, multi-scale feature extraction is performed on the basic features corresponding to the image to be processed to obtain the multi-scale features corresponding to the image to be processed.

[0008] In an optional implementation, the second network includes a fusion layer, a prediction layer, and a segmentation layer; Using the second network of the image segmentation model, based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed, a dynamic threshold is predicted and a probability map of the image to be processed is generated, including: Using the fusion layer, the wavelet transform features and multi-scale features corresponding to the image to be processed are fused to obtain the enhanced features corresponding to the image to be processed; Using the prediction layer, prediction is performed based on the enhanced features corresponding to the image to be processed to obtain a dynamic threshold map of the image to be processed; the dynamic threshold map represents the threshold of each region in the corresponding image. Using the segmentation layer, a probability map of the image to be processed is generated based on the enhanced features and dynamic threshold map corresponding to the image to be processed.

[0009] In an optional implementation, the image-text separation result of the image to be processed is obtained based on the probability map of the image to be processed, including: Connectivity analysis is performed on the probability graph of the image to be processed to obtain a mask image of the image to be processed, which is used as the image-text separation result of the image to be processed; the mask image represents the text region and non-text region in the corresponding image.

[0010] In an optional implementation, the image-text segmentation model is trained in the following manner: Data augmentation is performed on multiple original images to obtain multiple image samples and an annotation map is generated for each image sample; For each image sample, the image sample is preprocessed to obtain edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples. Based on the wavelet transform image samples, the wavelet frequency domain features corresponding to the image samples are obtained. Using the first network of the image segmentation model to be trained, multi-scale features corresponding to the image samples are generated based on the image samples, the edge image samples, the enhanced image samples, the wavelet transform image samples, and the repaired image samples. Using the second network of the image-text segmentation model to be trained, a dynamic threshold is predicted and a probability map of the image sample is generated based on the wavelet frequency domain features and multi-scale features corresponding to the image sample; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively; Based on the probability map of the image samples, the image-text separation result of the image samples is obtained; The image-text segmentation model is trained based on the labeled map, probability map, and image-text separation results of each image sample to obtain the image-text segmentation model.

[0011] In an optional implementation, the image samples are preprocessed to obtain edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples. Based on the wavelet transform image samples, the wavelet frequency domain features corresponding to the image samples are obtained, including: Edge extraction is performed on the image sample to obtain the edge image sample; The image sample is enhanced to obtain the enhanced image sample; Perform wavelet transform on the image samples to obtain the wavelet-transformed image samples; The image sample is repaired to obtain the repaired image sample; Feature extraction is performed on the wavelet transform image sample to obtain the wavelet frequency domain features corresponding to the image sample.

[0012] In an optional implementation, the image-text separation result of the image sample is a mask image representing the text region and non-text region in the image sample; Based on the annotation map, probability map, and image-text separation results of each image sample, the image-text segmentation model to be trained is trained to obtain the image-text segmentation model, including: Calculate the cross-entropy loss based on the labeled map and probability map of each image sample; Calculate the structural similarity loss based on the labeled image and its mask image for each image sample; Calculate the edge gradient loss based on the gradient of the labeled image and the gradient of the mask image for each image sample; The cross-union ratio loss is calculated based on the text region in the labeled image and the text region in the mask image of each image sample. The total loss is calculated based on the cross-entropy loss, the structural similarity loss, the edge gradient loss, and the cross-union ratio loss. The image-text segmentation model to be trained is then trained based on the total loss to obtain the image-text segmentation model.

[0013] Secondly, the present invention provides an image-text separation device, the device comprising: The preprocessing module is used to preprocess the image to be processed to obtain an edge image, an enhanced image, and a wavelet transform image, and to obtain the wavelet frequency domain features corresponding to the image to be processed based on the wavelet transform image. The segmentation module is used to generate multi-scale features corresponding to the image to be processed based on the first network of the image segmentation model, the image to be processed, the edge image, the enhanced image, and the wavelet transform image. Using the second network of the image-text segmentation model, a dynamic threshold is predicted and a probability map of the image to be processed is generated based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively; The post-processing module is used to obtain the image-text separation result of the image to be processed based on the probability map of the image to be processed.

[0014] Thirdly, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the image-text separation method described in any of the foregoing embodiments.

[0015] Fourthly, the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the image-text separation method described in any of the foregoing embodiments.

[0016] The image-text separation method, apparatus, electronic device, and storage medium provided in this invention include: firstly, preprocessing the image to be processed to obtain an edge image, an enhanced image, and a wavelet transform image; and obtaining the wavelet frequency domain features corresponding to the image to be processed based on the wavelet transform image; then, using a first network of the image-text segmentation model, generating multi-scale features corresponding to the image to be processed based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image; and using a second network of the image-text segmentation model, predicting a dynamic threshold and generating a probability map of the image to be processed based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed; the probability map represents the probability that each pixel in the corresponding image belongs to text or non-text; finally, obtaining the image-text separation result of the image to be processed based on the probability map of the image to be processed. This improves the accuracy and efficiency of image-text separation.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A block diagram of an electronic device provided in an embodiment of the present invention is shown; Figure 2 A schematic flowchart of the image-text separation method provided in an embodiment of the present invention is shown; Figure 3 An example diagram of the image-text separation method provided in an embodiment of the present invention is shown; Figure 4 A schematic diagram of the training process of the image-text segmentation model provided in an embodiment of the present invention is shown; Figure 5 A functional block diagram of the image-text separation device provided in an embodiment of the present invention is shown.

[0020] Icons: 100 - Electronic device; 110 - Processor; 120 - Memory; 130 - Communication module; 300 - Image and text separation device; 310 - Preprocessing module; 330 - Segmentation module; 350 - Postprocessing module; 370 - Training module. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] Image-text separation technology is an important research direction in the field of computer vision and image processing. Its core objective is to accurately extract text regions from complex images containing both text and non-text elements. With the acceleration of digitization, this technology plays a crucial role in scenarios such as document processing, invoice recognition, and archive digitization.

[0025] Traditional methods primarily achieve image-text separation by analyzing pixel grayscale value distribution, such as the Otsu method, Niblack method, and Sauvola method. These methods perform well under fixed backgrounds and uniform lighting conditions, but they are prone to misidentifying background noise as text when faced with complex textured backgrounds, non-uniform lighting, or scenes with mixed text, thus affecting separation accuracy. While deep learning-based methods for text detection and image-text separation exist, such as those using CNNs (Convolutional Neural Networks) and Transformers, these methods rely on fixed thresholds, leading to poor performance when handling diverse text types. Furthermore, while multi-scale feature extraction methods have been proposed to handle text regions of varying sizes, they still rely on globally fixed thresholds, making them unsuitable for complex scenarios. Therefore, this invention provides an image-text separation method that improves accuracy and efficiency through multi-scale feature extraction and dynamic threshold prediction.

[0026] Please see Figure 1 This is a block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes a processor 110, a memory 120, and a communication module 130. These components are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0027] The processor 110 is used to read / write data or programs stored in the memory 120 and perform corresponding functions. It can be a general-purpose processor, including CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be a DSP digital signal processor, ASIC application-specific integrated circuit, FPGA off-the-shelf programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0028] The memory 120 is used to store programs or data, and it can be RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0029] The communication module 130 is used for signaling or data communication with other devices.

[0030] Understandable, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0031] The above-mentioned electronic device will be used as the execution subject to execute the various steps of the methods provided in the embodiments of the present invention and achieve the corresponding technical effects.

[0032] Please see Figure 2 This is a schematic flowchart of the image-text separation method provided in the embodiments of the present invention.

[0033] Step S202: Preprocess the image to be processed to obtain the edge image, the enhanced image and the wavelet transform image, and obtain the wavelet frequency domain features corresponding to the image to be processed based on the wavelet transform image.

[0034] Step S204: Using the first network of the image segmentation model, multi-scale features corresponding to the image to be processed are generated based on the image to be processed, edge image, enhanced image, and wavelet transform image.

[0035] Step S206: Using the second network of the image-text segmentation model, based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed, predict the dynamic threshold and generate a probability map of the image to be processed; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively.

[0036] Step S208: Obtain the image-text separation result of the image to be processed based on the probability map of the image to be processed.

[0037] In this embodiment, the image-text separation method combines multi-source input fusion with a model to achieve high-precision image-text separation in complex backgrounds. The steps described above will be explained in detail below.

[0038] First, the image to be processed is acquired and preprocessed to obtain an edge image, an enhanced image, and a wavelet transform image, thus acquiring multiple feature information based on the image to be processed. Furthermore, based on the wavelet transform image, the wavelet frequency domain features corresponding to the image to be processed are obtained, so as to reflect the high-frequency details related to text in the image to be processed through these wavelet frequency domain features.

[0039] Then, using the first network of the image-text segmentation model, features at different scales are extracted based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image, thus obtaining the multi-scale features corresponding to the image to be processed. Next, using the second network of the image-text segmentation model, appropriate segmentation thresholds are dynamically predicted for each region in the image to be processed based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed, and a probability map is generated to determine the probability that each pixel in the image to be processed belongs to text and non-text respectively.

[0040] Finally, based on the probability map of the image to be processed, the text regions and non-text regions in the image to be processed are determined, and the image-text separation result of the image to be processed can be obtained.

[0041] In essence, this invention preprocesses images to obtain edge images, enhanced images, and wavelet transform images, providing the model with richer input data and thus improving the model's ability to perceive text regions. Furthermore, through multi-scale feature extraction and dynamic threshold prediction, the model can fully capture the spatial distribution characteristics of text regions and the semantic relationships of background information. This improves the accuracy and efficiency of image-text separation.

[0042] It is understood that step S202, the preprocessing of the image to be processed, is one of the key steps in the image-text separation method. Its purpose is to obtain rich input data by transforming the image to be processed in various ways to improve the segmentation effect of the subsequent model. Furthermore, step S202 provided in this embodiment of the invention includes steps S202-1 to S202-4, which will be described below.

[0043] Step S202-1: Extract edges from the image to be processed to obtain an edge image.

[0044] Step S202-2: Enhance the image to be processed to obtain an enhanced image.

[0045] Step S202-3: Perform wavelet transform on the image to be processed to obtain the wavelet transformed image.

[0046] Step S202-4: Extract features from the wavelet transform image to obtain the wavelet frequency domain features corresponding to the image to be processed.

[0047] In this embodiment, the main goal of preprocessing the image to be processed is to generate edge images, enhanced images, and wavelet transform images, and to extract relevant features from the wavelet transform images to provide multi-dimensional information support for subsequent models.

[0048] For ease of understanding, an example diagram is provided in this embodiment of the invention. Please refer to it. Figure 3 The following will combine Figure 3 Let's explain. First, we perform edge extraction on the image to be processed, resulting in an edge image. For example, this operation can be achieved using edge detection algorithms such as the Canny algorithm. The Canny algorithm can clearly capture the contours and boundary information in the image, thereby enhancing the model's sensitivity to the boundaries of text regions and improving the accuracy of segmentation through the edge image.

[0049] Next, enhancement operations are performed on the image to be processed to improve its quality, especially for images under low-light or blurry conditions. For example, this operation can employ techniques such as NLM (Non-Local Means), Retinex, SSR (Single Scale Retinex), MSR (Multi-Scale Retinex), MSRCR (Multi-Scale Retinex with Color Restoration), and HE (Histogram Equalization) to adjust brightness and optimize contrast, making text areas more prominent while suppressing background noise interference. This allows the enhanced image to provide clearer visual information to the model.

[0050] Next, wavelet transform is applied to the image to decompose its frequency domain characteristics. Wavelet transform can capture detailed information in the image, including high-frequency text edges and low-frequency background textures. Therefore, wavelet transform preserves the spatial structure of the image and enhances the representation of high-frequency features in text regions.

[0051] Finally, feature extraction is performed on the wavelet-transformed image to uncover the frequency domain characteristics related to the text, thus obtaining the wavelet frequency domain features corresponding to the image to be processed. For example, this operation can be performed by calculating the statistics of the wavelet coefficients or extracting energy distribution information within a specific frequency range to obtain the wavelet frequency domain features. Using wavelet frequency domain features can improve the model's ability to perceive text regions in complex backgrounds.

[0052] It can be understood that the embodiments of the present invention generate rich input data through various means such as edge extraction, image enhancement and wavelet transform, so as to help the model achieve high-precision image-text separation in complex images by leveraging the multi-dimensional characteristics of text.

[0053] It is understood that the first network in step S204 above is one of the core components of the image-text segmentation model, and its main function is to generate multi-scale features of the image to be processed. Furthermore, step S204 provided in this embodiment of the invention includes steps S204-1 to S204-2, which will be described below.

[0054] Step S204-1: Using the mapping layer, perform dimensional transformation on the image to be processed, the edge image, the enhanced image, and the wavelet transform image to obtain the basic features corresponding to the image to be processed.

[0055] Step S204-2: Using the extraction layer, multi-scale feature extraction is performed on the basic features corresponding to the image to be processed to obtain the multi-scale features corresponding to the image to be processed.

[0056] In this embodiment, the first grid of the image-text segmentation module includes a mapping layer and an extraction layer. The purpose of the first network is to integrate multiple input data from the preprocessing stage (including the image to be processed, edge image, enhanced image, and wavelet transform image) to generate multi-scale features that can reflect information at different scales through the synergistic effect of the mapping layer and the extraction layer, thereby improving the model's ability to understand text regions in complex scenes.

[0057] To facilitate understanding, we will continue with the following... Figure 3 The process is explained below. First, a mapping layer is used to perform dimensionality transformation on the image to be processed, the edge image, the enhanced image, and the wavelet-transformed image. Specifically, the mapping layer maps these multi-channel input data to the same feature dimension, thereby obtaining the basic features corresponding to the image to be processed. This process ensures that subsequent feature extraction operations can be performed in a consistent dimensional space, while preserving the original information of each input data. For example, the mapping layer can achieve dimensionality transformation through convolution operations (such as 1x1 convolution), making the feature representation of each input data more compact and adaptable to subsequent processing requirements.

[0058] Subsequently, an extraction layer is used to extract multi-scale features from the basic features, that is, to capture features at different scales from the basic features. For example, the extraction layer can adopt a network structure of Swin Transformer or RSTB (Residual Swin Transformer Block). This network structure includes a window attention mechanism and a local-global feature fusion strategy, which can effectively extract multi-level information from local details to global structure in the image. Thus, the obtained multi-scale features can comprehensively reflect the spatial distribution characteristics of the text region and the semantic association of background information.

[0059] In essence, this invention achieves an effective transformation from multi-source input data to multi-scale features through the collaborative work of the mapping layer and the extraction layer. The mapping layer is responsible for unifying input data from different sources to the same dimension, while the extraction layer focuses on mining feature information at different scales. This improves the model's adaptability and enhances its ability to perceive text regions in complex backgrounds.

[0060] It is understood that the second network in step S206 above is one of the core components of the image-text segmentation model. Its main function is to predict the dynamic threshold and generate a probability map based on the wavelet frequency domain features and multi-scale features of the image to be processed. Furthermore, step S206 provided in this embodiment of the invention includes steps S206-1 to S206-3, and these sub-steps will be described below.

[0061] Step S206-1: Using the fusion layer, the wavelet transform features and multi-scale features corresponding to the image to be processed are fused to obtain the enhanced features corresponding to the image to be processed.

[0062] Step S206-2: Using the prediction layer, prediction is performed based on the enhancement features corresponding to the image to be processed to obtain the dynamic threshold map of the image to be processed; the dynamic threshold map represents the threshold of each region in the corresponding image.

[0063] Step S206-3: Using the segmentation layer, a probability map of the image to be processed is generated based on the enhanced features and dynamic threshold map corresponding to the image to be processed.

[0064] In this embodiment, the second grid of the image-text segmentation module includes a fusion layer, a prediction layer, and a segmentation layer. The purpose of the second network is to generate a probability map that reflects the probability of each pixel belonging to text or non-text through the synergistic effect of the fusion layer, the prediction layer, and the segmentation layer, thereby improving the model's segmentation accuracy of text regions in complex scenes.

[0065] To facilitate understanding, we will continue with the following... Figure 3The explanation begins with the fusion layer. First, a fusion layer is used to fuse the wavelet transform features and multi-scale features of the image to be processed. Specifically, the fusion layer integrates feature information from different sources into enhanced features. For example, the fusion layer can use a channel attention mechanism to dynamically enhance important channels in the frequency domain, effectively fusing high-frequency wavelet information with multi-scale features. This results in enhanced features that more comprehensively reflect the spatial distribution characteristics of the text region and the semantic relationships of the background information.

[0066] Then, a prediction layer is used to predict based on the enhanced features, resulting in a dynamic threshold map of the image to be processed. Specifically, the prediction layer dynamically predicts suitable segmentation thresholds for each region in the image. For example, this process can be achieved through a convolutional attention mechanism, mapping the enhanced features to the dynamic threshold map. The dynamic threshold map can reflect segmentation standards at different spatial locations, thus meeting the need for accurate segmentation of text regions in complex backgrounds.

[0067] Finally, a segmentation layer generates a probability map of the image to be processed based on the enhanced features and the dynamic threshold map. Specifically, the segmentation layer comprehensively considers the information from the enhanced features and the dynamic threshold map to perform pixel-level binary classification, outputting the probability of each pixel belonging to text and non-text, so that the generated probability map can clearly reflect the distribution of text and non-text regions.

[0068] In essence, this invention achieves an efficient transformation from enhanced features to a dynamic threshold map and then to a probabilistic map through the collaborative work of the fusion layer, prediction layer, and segmentation layer. The fusion layer integrates feature information from different sources into enhanced features, the prediction layer focuses on generating the dynamic threshold map, and the segmentation layer ultimately generates the probabilistic map. This improves the model's ability to locate irregular text regions and enhances the segmentation effect on text regions in complex scenes.

[0069] Optionally, for step S208, this embodiment of the invention provides a possible implementation method, namely: performing connected component analysis on the probability graph of the image to be processed to obtain a mask image of the image to be processed, and using it as the image-text separation result of the image to be processed; the mask image represents the text region and non-text region in the corresponding image.

[0070] Understandably, the probabilistic map is generated by a model, representing the probability that each pixel belongs to text or non-text. However, the probabilistic map is not suitable as a direct result of image-text separation, so it needs further processing to obtain a clear division between text and non-text regions.

[0071] To facilitate understanding, we will continue with the following... Figure 3Let's explain. For example, the CCA (Connected Component Analysis) algorithm can be used to analyze the probability graph of the image to be processed. Specifically, the purpose of connected component analysis is to identify interconnected high-probability regions in the probability graph. These regions typically correspond to text regions. For instance, by comparing the probability of each pixel in the probability graph belonging to text and non-text, we can select each target pixel whose probability of belonging to text is greater than its probability of belonging to non-text. Connecting these high-probability regions formed by target pixels together identifies the text regions.

[0072] Subsequently, based on the text regions obtained from connected component analysis, a mask image is generated, resulting in the text-image separation of the image to be processed. The mask image is a binary image, where text regions are labeled with a first specific value, such as 1, and non-text regions are labeled with a second specific value, such as 0. This binary image clearly reflects the distribution of text and non-text regions. Furthermore, to improve the quality of the mask image, additional filtering operations can be added to the connected component analysis, such as removing connected components with excessively small areas to filter out potential noise interference.

[0073] Optionally, based on the text region obtained from connected component analysis, OCR (Optical Character Recognition) technology can also be used to recognize the text content of the text region, so as to meet other business needs such as document digitization.

[0074] In essence, this invention achieves a transformation from probabilistic representation to explicit region division by performing connected component analysis on the probabilistic graph to generate a mask image. This process not only preserves the accuracy of the model's predictions but also ensures that the final output is more intuitive for subsequent applications.

[0075] Optionally, based on the above-described image-text segmentation model, this embodiment of the invention also provides a training method for the image-text segmentation model. Please refer to [link to relevant documentation]. Figure 4 .

[0076] Step S210: Perform data augmentation on multiple original images to obtain multiple image samples and generate a labeled map for each image sample.

[0077] Step S212: For each image sample, preprocess the image sample to obtain edge image sample, enhanced image sample, wavelet transform image sample and repaired image sample, and obtain the wavelet frequency domain features corresponding to the image sample based on the wavelet transform image sample.

[0078] Step S214: Using the first network of the image segmentation model to be trained, multi-scale features corresponding to the image samples are generated based on the image samples, edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples.

[0079] Step S216: Using the second network of the image-text segmentation model to be trained, predict the dynamic threshold and generate a probability map of the image sample based on the wavelet frequency domain features and multi-scale features corresponding to the image sample; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively.

[0080] Step S218: Obtain the image-text separation result of the image sample based on the probability map of the image sample.

[0081] Step S220: Based on the annotation map, probability map and image-text separation results of each image sample, train the image-text segmentation model to be trained to obtain the image-text segmentation model.

[0082] In this embodiment, multiple original images can be obtained by acquiring publicly available image datasets, such as ICDAR, SynthText, and MSRA-TD500. These original images should not only contain rich mixed image and text scenes but also meet the requirements of scene changes under different lighting conditions and complex backgrounds. Furthermore, to improve the diversity of training data and enhance the model's adaptability, data augmentation can be performed on the original images. For example, transformations such as rotation, cropping, adding noise, adjusting brightness and contrast, occlusion, and blurring can be used to obtain multiple image samples.

[0083] Rotation refers to randomly rotating the image. Cropping refers to randomly cropping regions of different sizes from the image. Adding noise refers to enhancing image noise by adding Gaussian noise, salt-and-pepper noise, etc., to improve the model's robustness to noise interference. Adjusting brightness and contrast is used to simulate image changes under different lighting conditions. Occlusion and blurring are used to simulate situations where text is occluded or blurred, to improve the model's performance in practical applications.

[0084] Furthermore, multiple image samples are annotated to obtain an annotated image for each sample. For example, pixels in the text region of an image sample are annotated with a first specific value (1), and pixels in the non-text region of the same image sample are annotated with a second specific value (0), thus obtaining the annotated image for that image sample. Optionally, for image samples that are more difficult to annotate, such as those with low contrast or high noise, annotated images can be generated by setting a threshold or using auxiliary annotation methods.

[0085] Then, for each image sample, preprocessing operations are performed to generate edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples, and the corresponding wavelet frequency domain features are extracted from the wavelet transform image samples.

[0086] Next, using the first network of the image-text segmentation model, features at different scales are extracted from image samples, edge image samples, enhanced image samples, and wavelet transform image samples, thus obtaining the multi-scale features corresponding to the image samples. Then, using the second network of the image-text segmentation model, appropriate segmentation thresholds are dynamically predicted for each region in the image sample based on the wavelet frequency domain features and multi-scale features corresponding to the image samples, and a probability map is generated to determine the probability that each pixel in the image sample belongs to text or non-text. Furthermore, based on the probability map of the image samples, the text regions and non-text regions in the image samples are determined, thus obtaining the image-text separation result of the image samples.

[0087] By processing each image sample in a similar manner, we can obtain the probability map and image-text separation result for each image sample. By combining the labeled map of each image sample with the image-text segmentation model to be trained, we can obtain the image-text segmentation model.

[0088] For example, 80% of the image samples in the image sample set are used as the training set, and 20% are used as the validation set. The training set is divided into multiple batches according to a preset batch size, such as 16. Then, following a preset number of training epochs, such as 500 epochs, and an early stopping strategy (stopping training when validation accuracy no longer improves), the image segmentation model is trained using the training set and validated using the validation set. Simultaneously, the initial learning rate is set to 1e-4, and the learning rate is gradually reduced through learning rate decay, i.e., exponential decay based on validation accuracy or cosine annealing. Furthermore, a warm-up strategy is used, gradually increasing the learning rate in the first 10 epochs to avoid unstable parameter updates and oscillations in the early stages of model learning. The AdamW optimizer is also used to prevent overfitting and achieve stable convergence.

[0089] It can be understood that the training process of this embodiment of the invention ranges from data augmentation to multi-source feature extraction, and then to dynamic threshold prediction and probabilistic graph generation, in order to help the model learn high-precision image and text segmentation capabilities from multiple dimensions, thereby enabling the model to achieve highly robust image and text separation tasks in complex backgrounds.

[0090] Optionally, for step S212, this embodiment of the invention provides a possible implementation.

[0091] Step S212-1: Extract edges from the image samples to obtain edge image samples.

[0092] Step S212-2: Enhance the image sample to obtain an enhanced image sample.

[0093] Step S212-3: Perform wavelet transform on the image samples to obtain wavelet transformed image samples.

[0094] Step S212-4: Repair the image sample to obtain the repaired image sample.

[0095] Step S212-5: Extract features from the wavelet transform image samples to obtain the wavelet frequency domain features corresponding to the image samples.

[0096] In this embodiment, the main goal of preprocessing the image samples is to generate edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples, and to extract relevant features from the wavelet transform image samples to provide accurate and multi-dimensional information for subsequent model training.

[0097] It is understood that edge extraction is performed on image samples to obtain edge image samples, enhancement is performed on image samples to obtain enhanced image samples, wavelet transform is performed on image samples to obtain wavelet transformed image samples, and the corresponding wavelet frequency domain features are extracted from the wavelet transformed image samples. These implementation processes are similar to the processing method for the image to be processed described above, and their basic principles and resulting technical effects are the same as in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.

[0098] It should be noted that during the model training phase, this embodiment of the invention will also perform blur detection and repair operations on the image samples to improve the quality of the image samples, so as to help the model learn image and text segmentation capabilities through the obtained repaired image samples.

[0099] Optionally, for step S220, this embodiment of the invention provides a possible implementation.

[0100] Step S220-1: Calculate the cross-entropy loss based on the labeled map and probability map of each image sample. Step S220-2: Calculate the structural similarity loss based on the labeled image and mask image of each image sample. Step S220-3: Calculate the edge gradient loss based on the gradient of the labeled image and the gradient of the mask image for each image sample.

[0101] Step S220-4: Calculate the cross-union ratio loss based on the text region in the annotation image of each image sample and the text region in its mask image.

[0102] Step S220-5: Calculate the total loss based on the cross-entropy loss, structural similarity loss, edge gradient loss, and cross-union ratio loss values, and train the image-text segmentation model to be trained based on the total loss to obtain the image-text segmentation model.

[0103] In this embodiment, the cross-entropy loss, structural similarity loss, edge gradient loss, and intersection-union ratio loss of each batch group can be calculated, and the total loss of each batch group can be calculated. Then, the total loss of each batch group can be used to train the image-text segmentation model to be trained, thus obtaining the image-text segmentation model.

[0104] Understandably, the loss value of each image sample in a batch is calculated in a similar way. For ease of understanding, the following explanation uses an image sample as an example.

[0105] First, according to the preset cross-entropy loss function, the cross-entropy loss value of the image sample is calculated based on the labeled map and probability map of the image sample. The cross-entropy loss function is as follows: ; in, This represents the cross-entropy loss value of the image sample; The label value represents the i-th pixel in the labeled image sample. The label value is either a first specific value, i.e., 1, or a second specific value, i.e., 0. The probability of the i-th pixel in the probability map of the image sample belonging to text is represented by N; N represents the total number of pixels in the labeled map of the image sample, and the labeled map has the same number of pixels as the mask image.

[0106] Cross-entropy loss can be understood as measuring the difference between the probability map predicted by the model and the true labeled map, with the aim of optimizing the accuracy of pixel-level classification tasks. For example, for each pixel, if it belongs to text (i.e., its label value is the first specific value), and the model predicts a lower probability value, it will lead to a larger cross-entropy loss. By minimizing the cross-entropy loss, the model can gradually improve its ability to distinguish between text and non-text regions.

[0107] Then, according to the preset structural similarity loss function, the structural similarity loss value of the image sample is calculated based on the labeled image and the mask image of the image sample. The structural similarity loss function is as follows: ; in, This represents the structural similarity loss value of image samples; A labeled map representing image samples; A mask image representing an image sample; This represents the structural similarity operation.

[0108] Structural similarity loss can be understood as an evaluation of the structural consistency between the generated mask image and the real labeled image. It improves the visual quality of the segmentation result by comparing the overall structural features of the image and optimizing the background reconstruction. For example, when the generated mask image has obvious structural distortions or missing parts in some areas, the structural similarity loss will increase significantly, prompting the model to adjust its parameters to improve the performance in these areas.

[0109] Next, edge detection algorithms such as the Canny algorithm are used to perform edge detection on both the labeled image and the mask image of the image sample. The gradients are then calculated based on the edge detection results of both, resulting in the labeled image gradient and the mask image gradient. The labeled image gradient includes the gradient value of each pixel in the labeled image, and the mask image gradient includes the gradient value of each pixel in the mask image. Furthermore, according to a preset edge gradient loss function, the edge gradient loss value is calculated based on the labeled image gradient and the mask image gradient of the image sample. The edge gradient loss function is shown below: ; in, This represents the edge gradient loss value of the image sample; This represents the gradient value of the i-th pixel in the labeled image of the image sample. represents the gradient value of the i-th pixel in the mask image of the image sample; N represents the total number of pixels in the annotation image of the image sample, and the annotation image has the same number of pixels as the mask image.

[0110] This can be understood as edge gradient loss optimizing the model's ability to segment text edges. It emphasizes accurate characterization of text boundary regions by comparing the gradient information of the generated mask image with that of the actual labeled image. For example, if the model produces blurry or incorrect segmentation results at text edges, it will lead to a large edge gradient loss. By minimizing the edge gradient loss, the model can better capture the contour information of text regions against complex backgrounds.

[0111] Then, according to the preset Cross-Union-Ratio (CURRR) loss function, the CURRR value is calculated based on the text regions in the labeled image of the image sample and the text regions in the mask image. The CURRR loss function is as follows: ; in, A represents the cross-union ratio loss value of the image sample; B represents the set of pixels in the text region of the labeled image of the image sample; C represents the set of pixels in the text region of the masked image of the image sample.

[0112] The intersection-union ratio (IURR) loss can be understood as a measure of the overlap between the text regions predicted by the model and the actual text regions. It optimizes the overall consistency of the segmentation results by calculating the ratio of the intersection to the union of the two regions. For example, when the text regions predicted by the model are too large or too small, the IURR loss will increase significantly, prompting the model to adjust its parameters to locate the text regions more accurately.

[0113] Finally, based on the cross-entropy loss values ​​of all image samples in the batch, the cross-entropy loss of the batch is calculated. Based on the structural similarity loss values ​​of all image samples in the batch group, calculate the structural similarity loss of the batch group. Based on the edge gradient loss values ​​of all image samples in the batch group, calculate the edge gradient loss of the batch group. Based on the cross-union ratio (CUP) loss values ​​of all image samples in the batch group, calculate the CUP loss of the batch group. Then, according to the preset weighting formula, based on the cross-entropy loss of the batch group, i.e. Structural similarity loss Marginal gradient loss Cross-combination loss Calculate the total loss for the batch group, and the weighting formula is as follows: ; in, This indicates the total loss for the batch group; This represents the cross-entropy loss of the batch group; This represents the structural similarity loss of the batch group; This represents the edge gradient loss of the batch group; Indicates the crossover ratio loss of the batch group; This represents the first weighting coefficient, for example... ; This represents the second weighting coefficient, for example... ; This represents the third weighting coefficient, for example... ; This represents the fourth weighting coefficient, for example .

[0114] In essence, the training process of this invention improves the model's segmentation performance from multiple dimensions through the combined optimization of multiple loss functions. Cross-entropy loss focuses on pixel-level classification accuracy, structural similarity loss optimizes background reconstruction quality, edge gradient loss improves the clarity of text edges, and intersection-union ratio (IUU) loss enhances the consistency of the overall region. This strengthens the model's adaptability to complex scenes while ensuring the segmentation results offer dual advantages in both visual appeal and quantitative metrics.

[0115] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of an image-text separation device is given below. Please refer to... Figure 5 This is a functional block diagram of the image-text separation device 300 provided in this embodiment of the invention. It should be noted that the basic principle and technical effects of the image-text separation device 300 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The image-text separation device 300 includes: The preprocessing module 310 is used to preprocess the image to be processed to obtain an edge image, an enhanced image, and a wavelet transform image, and to obtain the wavelet frequency domain features corresponding to the image to be processed based on the wavelet transform image.

[0116] The segmentation module 330 is used to generate multi-scale features corresponding to the image to be processed based on the first network of the image-text segmentation model, the image to be processed, the edge image, the enhanced image, and the wavelet transform image; and to predict the dynamic threshold and generate a probability map of the image to be processed based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed using the second network of the image-text segmentation model; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively.

[0117] The post-processing module 350 is used to obtain the image-text separation result of the image to be processed based on the probability map of the image to be processed.

[0118] Optionally, the preprocessing module 310 is further configured to: extract edges from the image to be processed to obtain an edge image; enhance the image to be processed to obtain an enhanced image; perform wavelet transform on the image to be processed to obtain a wavelet transformed image; and extract features from the wavelet transformed image to obtain wavelet frequency domain features corresponding to the image to be processed.

[0119] Optionally, the segmentation module 330 is further configured to: utilize the mapping layer to perform dimensionality transformation on the image to be processed, the edge image, the enhanced image, and the wavelet transform image to obtain the basic features corresponding to the image to be processed; and utilize the extraction layer to perform multi-scale feature extraction on the basic features corresponding to the image to be processed to obtain the multi-scale features corresponding to the image to be processed.

[0120] Optionally, the segmentation module 330 is further configured to: use a fusion layer to fuse wavelet transform features and multi-scale features corresponding to the image to be processed to obtain enhanced features corresponding to the image to be processed; use a prediction layer to predict based on the enhanced features corresponding to the image to be processed to obtain a dynamic threshold map of the image to be processed; the dynamic threshold map represents the threshold of each region in the corresponding image; and use a segmentation layer to generate a probability map of the image to be processed based on the enhanced features and dynamic threshold map corresponding to the image to be processed.

[0121] Optionally, the post-processing module 350 is further configured to: perform connected component analysis on the probability graph of the image to be processed to obtain a mask image of the image to be processed, and use it as the image-text separation result of the image to be processed; the mask image represents the text region and non-text region in the corresponding image.

[0122] Optionally, the image-text separation device 300 further includes a training module 370, used for: performing data augmentation on multiple original images to obtain multiple image samples and generating a labeled map for each image sample; for each image sample, preprocessing the image sample to obtain edge image samples, augmented image samples, wavelet transform image samples, and repaired image samples, and obtaining wavelet frequency domain features corresponding to the image sample based on the wavelet transform image samples; using the first network of the image-text segmentation model to be trained, generating multi-scale features corresponding to the image sample based on the image samples, edge image samples, augmented image samples, wavelet transform image samples, and repaired image samples; using the second network of the image-text segmentation model to be trained, predicting a dynamic threshold and generating a probability map of the image sample based on the wavelet frequency domain features and multi-scale features corresponding to the image sample; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively; obtaining the image-text separation result of the image sample based on the probability map of the image sample; and training the image-text segmentation model to be trained based on the labeled map, probability map, and image-text separation result of each image sample to obtain the image-text segmentation model.

[0123] Optionally, the training module 370 is also used to: extract edges from image samples to obtain edge image samples; enhance image samples to obtain enhanced image samples; perform wavelet transform on image samples to obtain wavelet transformed image samples; repair image samples to obtain repaired image samples; and extract features from wavelet transformed image samples to obtain wavelet frequency domain features corresponding to the image samples.

[0124] Optionally, the training module 370 is further configured to: calculate cross-entropy loss based on the annotation map and its probability map of each image sample; calculate structural similarity loss based on the annotation map and its mask image of each image sample; calculate edge gradient loss based on the gradient of the annotation map and its mask image of each image sample; calculate intersection-union ratio (IUR) loss based on the text region in the annotation map and the text region in the mask image of each image sample; calculate the total loss based on the cross-entropy loss, structural similarity loss, edge gradient loss, and IUR loss values; and train the image-text segmentation model to be trained based on the total loss to obtain the image-text segmentation model.

[0125] This invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the image-text separation method disclosed in this invention.

[0126] This invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the image-text separation method disclosed in this invention.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0128] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0129] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of text and graphics separation, characterized by, The method includes: The image to be processed is preprocessed to obtain an edge image, an enhanced image, and a wavelet transform image. Based on the wavelet transform image, the wavelet frequency domain features corresponding to the image to be processed are obtained. Using a first network of an image segmentation model, multi-scale features corresponding to the image to be processed are generated based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image. The first network includes a mapping layer and an extraction layer. The second network of the image-text segmentation model predicts a dynamic threshold and generates a probability map of the image to be processed based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed. The probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively. The second network includes a fusion layer, a prediction layer and a segmentation layer. Based on the probability map of the image to be processed, the image-text separation result of the image to be processed is obtained; Using the first network of the image segmentation model, multi-scale features corresponding to the image to be processed are generated based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image, including: Using the mapping layer, the dimensionality transformation is performed on the image to be processed, the edge image, the enhanced image, and the wavelet transform image to obtain the basic features corresponding to the image to be processed; Using the extraction layer, multi-scale feature extraction is performed on the basic features corresponding to the image to be processed to obtain the multi-scale features corresponding to the image to be processed; Using the second network of the image segmentation model, based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed, a dynamic threshold is predicted and a probability map of the image to be processed is generated, including: Using the fusion layer, the wavelet transform features and multi-scale features corresponding to the image to be processed are fused to obtain the enhanced features corresponding to the image to be processed; Using the prediction layer, prediction is performed based on the enhanced features corresponding to the image to be processed to obtain a dynamic threshold map of the image to be processed; the dynamic threshold map represents the threshold of each region in the corresponding image. Using the segmentation layer, a probability map of the image to be processed is generated based on the enhanced features and dynamic threshold map corresponding to the image to be processed.

2. The method of claim 1, wherein, The image to be processed is preprocessed to obtain an edge image, an enhanced image, and a wavelet transform image. Based on the wavelet transform image, the wavelet frequency domain features corresponding to the image to be processed are obtained, including: Edge extraction is performed on the image to be processed to obtain the edge image; The image to be processed is enhanced to obtain the enhanced image; Perform wavelet transform on the image to be processed to obtain the wavelet transformed image; Feature extraction is performed on the wavelet transform image to obtain the wavelet frequency domain features corresponding to the image to be processed.

3. The method of claim 1, wherein the step of separating the text and the image comprises: Based on the probability map of the image to be processed, the image-text separation result of the image to be processed is obtained, including: Connectivity analysis is performed on the probability graph of the image to be processed to obtain a mask image of the image to be processed, which is used as the image-text separation result of the image to be processed; the mask image represents the text region and non-text region in the corresponding image.

4. The method of claim 1, wherein the step of separating the text and the image comprises the steps of: The image-text segmentation model was trained in the following manner: ​ Data augmentation is performed on multiple original images to obtain multiple image samples and an annotation map is generated for each image sample; For each image sample, the image sample is preprocessed to obtain edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples. Based on the wavelet transform image samples, the wavelet frequency domain features corresponding to the image samples are obtained. Using the first network of the image segmentation model to be trained, multi-scale features corresponding to the image samples are generated based on the image samples, the edge image samples, the enhanced image samples, the wavelet transform image samples, and the repaired image samples. Using the second network of the image-text segmentation model to be trained, a dynamic threshold is predicted and a probability map of the image sample is generated based on the wavelet frequency domain features and multi-scale features corresponding to the image sample; the probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively; Based on the probability map of the image samples, the image-text separation result of the image samples is obtained; The image-text segmentation model is trained based on the labeled map, probability map, and image-text separation results of each image sample to obtain the image-text segmentation model.

5. The method of texturizing according to claim 4, wherein, The image samples are preprocessed to obtain edge image samples, enhanced image samples, wavelet transform image samples, and repaired image samples. Based on the wavelet transform image samples, the wavelet frequency domain features corresponding to the image samples are obtained, including: Edge extraction is performed on the image sample to obtain the edge image sample; The image sample is enhanced to obtain the enhanced image sample; Perform wavelet transform on the image samples to obtain the wavelet-transformed image samples; The image sample is repaired to obtain the repaired image sample; Feature extraction is performed on the wavelet transform image sample to obtain the wavelet frequency domain features corresponding to the image sample.

6. The method of texturizing according to claim 4, wherein, The image-text separation result of the image sample is a mask image representing the text region and non-text region in the image sample; Based on the annotation map, probability map, and image-text separation results of each image sample, the image-text segmentation model to be trained is trained to obtain the image-text segmentation model, including: Calculate the cross-entropy loss based on the labeled map and probability map of each image sample; Calculate the structural similarity loss based on the labeled image and its mask image for each image sample; Calculate the edge gradient loss based on the gradient of the labeled image and the gradient of the mask image for each image sample; The cross-union ratio loss is calculated based on the text region in the labeled image and the text region in the mask image of each image sample. The total loss is calculated based on the cross-entropy loss, the structural similarity loss, the edge gradient loss, and the cross-union ratio loss. The image-text segmentation model to be trained is then trained based on the total loss to obtain the image-text segmentation model.

7. A text-image separation device, characterized in that, The device includes: The preprocessing module is used to preprocess the image to be processed to obtain an edge image, an enhanced image, and a wavelet transform image, and to obtain the wavelet frequency domain features corresponding to the image to be processed based on the wavelet transform image. The segmentation module is used to generate multi-scale features corresponding to the image to be processed based on the image to be processed, the edge image, the enhanced image, and the wavelet transform image using the first network of the image segmentation model. The first network includes a mapping layer and an extraction layer. The second network of the image-text segmentation model predicts a dynamic threshold and generates a probability map of the image to be processed based on the wavelet frequency domain features and multi-scale features corresponding to the image to be processed. The probability map represents the probability that each pixel in the corresponding image belongs to text and non-text respectively. The second network includes a fusion layer, a prediction layer and a segmentation layer. The post-processing module is used to obtain the image-text separation result of the image to be processed based on the probability map of the image to be processed; The segmentation module is specifically used to: utilize the mapping layer to perform dimensionality transformation on the image to be processed, the edge image, the enhanced image, and the wavelet transform image to obtain the basic features corresponding to the image to be processed; and utilize the extraction layer to perform multi-scale feature extraction on the basic features corresponding to the image to be processed to obtain the multi-scale features corresponding to the image to be processed. The segmentation module is further configured to: fuse wavelet transform features and multi-scale features corresponding to the image to be processed using the fusion layer to obtain enhanced features corresponding to the image to be processed; predict based on the enhanced features corresponding to the image to be processed using the prediction layer to obtain a dynamic threshold map of the image to be processed; the dynamic threshold map represents the threshold of each region in the corresponding image; and generate a probability map of the image to be processed using the segmentation layer based on the enhanced features and dynamic threshold map corresponding to the image to be processed.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and when the processor executes the computer program, it implements the image-text separation method according to any one of claims 1-6.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the image-text separation method according to any one of claims 1-6.

Images