Text Printing Quality Evaluation Method Based on Text Mask Structure Similarity

Abstract

The present invention discloses a method for evaluating text printing quality based on text mask structural similarity: a text mask-based dynamic perceptual text segmentation network is used to process printed text images and standard text images to obtain a multi-channel text fine segmentation feature map; an image refinement algorithm is used to extract text skeleton information from the multi-channel text fine segmentation feature map, and different weights are assigned to each layer of pixels based on the order in which pixels are removed during the extraction process to establish a text skeleton template; a structural similarity image quality evaluation algorithm is used to perform text printing quality detection on the standard text skeleton template and the printed text skeleton template, obtaining an average structural similarity value between the printed text skeleton template and the standard text skeleton template; and the text printing quality is determined based on the average structural similarity value. The present invention constructs a text segmentation network, and the output is a feature map containing text mask information, thereby improving the detection range and accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of data processing, and in particular relates to a text printing quality evaluation method based on text mask structure similarity. Background Art

[0002] With the continuous improvement of my country's industrialization level, printing technology has also been continuously improving and developing. Nowadays, more and more people cannot do without printed materials, and people's requirements for printing quality are getting higher and higher. Printing quality is not only an important indicator to measure the quality of printed products, but also a key link in the entire printing production.

[0003] In the real-world printing industry, printed images originate from all over life and may contain text instances of arbitrary structures, such as multi-directional, curved, and multilingual text. This undoubtedly increases the difficulty of network text detection, which in turn affects the quality of printed text. Traditional printing quality inspection methods do not evaluate text printing quality separately, and therefore have inevitable flaws in accuracy, stability, and detection range. Furthermore, traditional printing quality inspection methods, when binarizing image text, consider both the overall characteristics of the incomplete image and the local information of the text, which can easily lead to text stroke breakage and false strokes, resulting in poor detection accuracy. Therefore, proposing an evaluation method to improve and perfect text printing quality has become an urgent need in printing production. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method for evaluating text printing quality based on text mask structure similarity. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] A text printing quality evaluation method based on text mask structure similarity, the text printing quality evaluation method based on text mask structure similarity comprising:

[0006] A dynamic perceptual text segmentation network based on text masks is constructed to process printed text images and standard text images to obtain a multi-channel text fine segmentation feature map; the printed text image and the standard text image have the same image content, the dynamic perceptual text segmentation network based on text masks is pre-trained using a text image dataset with real labels, and the multi-channel text fine segmentation feature map carries semantic mask information;

[0007] Using an image thinning algorithm to extract text skeleton information from the multi-channel text fine segmentation feature map, assigning different weights to each layer of pixels according to the order in which pixels are removed during the extraction process, and establishing a text skeleton template; the text skeleton template includes a printed text skeleton template and a standard text skeleton template;

[0008] Using a structural similarity image quality evaluation algorithm, the standard text skeleton template and the printed text skeleton template are subjected to text printing quality detection to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template;

[0009] The text printing quality is determined according to the average structural similarity value.

[0010] In one embodiment of the present invention, the text mask-based dynamic perceptual text segmentation network includes an encoding network, a multi-level feature fusion module, and a decoding network;

[0011] The dynamic perceptual text segmentation network based on text mask is constructed to process printed text images and standard text images to obtain multi-channel text fine segmentation feature maps, including:

[0012] Performing feature extraction and feature encoding on the printed text image and the standard text image through the encoding network to obtain an encoded multi-channel feature map;

[0013] Through the multi-level feature fusion module, the encoded multi-channel feature map is processed including feature reconstruction, splicing and feature mapping to complete the feature aggregation function and obtain the rough segmentation result map F coarse and semantic feature vector F semantics ;

[0014] Through the decoding network, combined with the semantic feature vector F semantics For the coarse segmentation result image F coarse Feature enhancement is performed to generate the multi-channel text fine segmentation feature map.

[0015] In one embodiment of the present invention, the multi-level feature fusion module includes a feature enhancement module, a Concat layer, and a nonlinear feature mapping module;

[0016] The multi-level feature fusion module performs feature reconstruction, splicing and feature mapping on the encoded multi-channel feature map to complete the feature aggregation function and obtain a rough segmentation result map F. coarse and semantic feature vector F semantics ,include:

[0017] Performing feature reconstruction on the encoded multi-channel feature map through the feature enhancement module, and inputting the reconstructed feature map into the Concat layer;

[0018] Using the Concat layer, the reconstructed feature maps are spliced by channel to obtain a spliced feature map;

[0019] The nonlinear feature mapping module is used to perform feature mapping on the spliced feature map to complete the feature aggregation function and obtain the rough segmentation result map F. coarse and semantic feature vector F semantics .

[0020] In one embodiment of the present invention, the method of extracting text skeleton information from the multi-channel text fine segmentation feature map using an image thinning algorithm, assigning different weights to each layer of pixels according to the order in which pixels are removed during the extraction process, and establishing a text skeleton template includes:

[0021] The multi-channel text fine segmentation feature map is scanned pixel by pixel using the ZS thinning algorithm, and the target pixels whose operation results meet the preset conditions are removed by performing arithmetic and logical operations on the eight neighborhoods of the target pixel to obtain the text skeleton of the multi-channel text fine segmentation feature map;

[0022] Different weights are assigned to pixels in each layer according to the order in which the pixels are removed, and a text skeleton template is established based on the weights.

[0023] In one embodiment of the present invention, the use of a structural similarity image quality assessment algorithm to perform text printing quality detection on the standard text skeleton template and the printed text skeleton template to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template includes:

[0024] Design a window of a specified size, move the window in pixels each time, calculate the structural similarity value of the image in each window, and obtain multiple local structural similarity values;

[0025] The multiple local structural similarity values are averaged to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template.

[0026] In one embodiment of the present invention, pre-training the text mask-based dynamic perceptual text segmentation network includes:

[0027] Acquire a training text image sample, where the training text image sample is a text image with a true label, and the true label carries character and semantic mask information in the training text image sample;

[0028] The text mask-based dynamic perceptual text segmentation network is trained using the training text image samples.

[0029] In one embodiment of the present invention, the step of training the text mask-based dynamic perceptual text segmentation network using the training text image samples includes:

[0030] Processing the training text image sample to obtain a predicted label;

[0031] Calculate the loss value of the predicted label and its corresponding true label;

[0032] Based on the loss value, the text mask-based dynamic perceptual text segmentation network is iteratively trained by adopting a gradient descent method.

[0033] In one embodiment of the present invention, calculating the loss value between the predicted label and its corresponding true label includes:

[0034] The loss value is calculated by the following formula:

[0035] L=L coarse +αL refine +βL reg ;

[0036] L coarse =CrossEntropy(F coarse ,F label );

[0037] L refine =CrossEntropy(F refine ,F label );

[0038] L reg =CrossEntropy(F reg ,F label );

[0039] Among them, L coarse is the loss value of the text coarse segmentation map, L refine is the loss value of the multi-channel text fine segmentation feature map, L reg For the rough segmentation graph F from the text coarse The character recognition loss value is obtained by cutting the corresponding sub-image area and feeding it into the character recognition network; CrossEntropy(·) is the cross entropy loss function, F label is the true label value, F refine is the multi-channel text fine segmentation feature map, F reg is the result value of the character recognition network, and α and β are preset values.

[0040] In one embodiment of the present invention, the decoding network is an object context representation network (OCR).

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention provides a text printing quality evaluation method based on text mask structural similarity. The method comprises: processing a printed text image and a standard text image by constructing a dynamic perceptual text segmentation network based on text masks to obtain a multi-channel text fine segmentation feature map, wherein the printed text image has the same image content as the standard text image, the dynamic perceptual text segmentation network based on text masks is obtained by pre-training a text image data set with real labels, and the multi-channel text fine segmentation feature map carries semantic mask information; extracting text skeleton information from the multi-channel text fine segmentation feature map by using an image refinement algorithm, assigning different weights to each layer of pixels according to the order in which pixels are removed during the extraction process, and establishing a text skeleton template, wherein the text skeleton template includes a printed text skeleton template and a standard text skeleton template; then using a structural similarity image quality evaluation algorithm to perform text printing quality detection on the standard text skeleton template and the printed text skeleton template to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template; and finally judging the text printing quality according to the average structural similarity value. The dynamic perceptual text segmentation network based on text mask constructed by the present invention outputs a multi-channel text precise segmentation feature map with text mask information, that is, the pixels in the text area have mask information, so that the output result can avoid the defects of incomplete text strokes, broken strokes, unclear edges, etc. caused by the complex background in the ordinary binarization process of traditional technology, thereby improving the detection accuracy; and by constructing a dynamic perceptual text segmentation network based on text mask, it can solve the problems of text instances of arbitrary structures that may exist in the input image and the large difference in pixel texture between different text instances, thereby improving the detection range and detection accuracy.

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a method for evaluating text printing quality based on text mask structure similarity provided by an embodiment of the present invention;

[0045] Figure 2-1 1 is a schematic diagram of a dynamic perceptual text segmentation network structure based on text mask provided by an embodiment of the present invention;

[0046] Figure 2-2 Schematic diagram of a structural similarity image quality assessment model provided by an embodiment of the present invention;

[0047] Figure 3 1 is a schematic diagram of a training process of a dynamic perceptual text segmentation network based on text mask provided by an embodiment of the present invention;

[0048] Figure 4This is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0050] Figure 1 FIG. 1 is a flow chart of a method for evaluating text printing quality based on text mask structure similarity provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0051] S100, processing printed text images and standard text images through the constructed dynamic perception text segmentation network based on text masks to obtain a multi-channel text precise segmentation feature map.

[0052] The printed text image and the standard text image have the same image content. The printed text image is a printed text image, while the standard text image can be an electronic version or an original image. The dynamic text segmentation network based on text masks is pre-trained using a text image dataset with real labels. The real labels carry the characters in the corresponding text image and semantic mask information. The semantic mask information can be information such as image texture or object category. Obviously, the multi-channel text segmentation feature map output by the dynamic text segmentation network based on text masks also carries semantic mask information.

[0053] Furthermore, in this embodiment, the dynamic perceptual text segmentation network based on text mask includes an encoding network, a multi-level feature fusion module, and a decoding network. When the dynamic perceptual text segmentation network based on text mask is constructed and used to process printed text images and standard text images to obtain a multi-channel text fine segmentation feature map:

[0054] Through the encoding network, feature extraction and feature encoding are performed on the printed text image and the standard text image to obtain the encoded multi-channel feature map;

[0055] Through the multi-level feature fusion module, the encoded multi-channel feature map is processed including feature reconstruction, splicing and feature mapping to complete the feature aggregation function and obtain the coarse segmentation result map F coarse and semantic feature vector F semantics ;

[0056] Through the decoding network, combined with the semantic feature vector F semantics For the coarse segmentation result image F coarse Perform feature enhancement to generate multi-channel text segmentation feature maps.

[0057] Specifically, in this embodiment, the multi-level feature fusion module includes: a feature enhancement module, a Concat layer, and a nonlinear feature mapping module; through the multi-level feature fusion module, the encoded multi-channel feature map is processed including feature reconstruction, splicing and feature mapping to complete the feature aggregation function and obtain a coarse segmentation result map F coarse and semantic feature vector F semantics hour:

[0058] The encoded multi-channel feature map is reconstructed through a feature enhancement module, and the reconstructed feature map is input into the Concat layer;

[0059] Use the Concat layer to splice the reconstructed feature map by channel to obtain the spliced feature map;

[0060] Through the nonlinear feature mapping module, the feature map after splicing is mapped to complete the feature aggregation function and obtain the rough segmentation result map F coarse and semantic feature vector F semantics .

[0061] In this embodiment, the structure of the multi-level feature fusion module can be set as: bidirectional attention module → cascade of 4 RCA modules → Concat layer → first convolution layer → pooling layer → batch normalization layer. HRNet can be used as the encoding network to perform feature extraction and feature encoding on printed text images and standard text images to obtain encoded multi-channel feature maps; then, through the feature enhancement module, the bidirectional attention mechanism (Dual Attention Mechanism, DAM) is used to reconstruct the high-level feature maps in the encoded multi-channel feature map, and the cascade region context region reconstruction module (Cascade Region Context Reassembly, CRCA) is used to reduce the fitting difficulty of the network in the large-scale sampling process, thereby improving the network's perception of text instance areas. After all the encoded multi-channel feature maps are upsampled, the encoded multi-channel feature maps of each layer are spliced by channel through the Concat layer, and the spliced feature maps are feature mapped to the combined features through a nonlinear feature mapping module composed of a convolution layer, a pooling layer, and a batch normalization layer to complete the feature aggregation function and obtain a coarse segmentation result map F. coarse and semantic feature vector F semantics .

[0062] The above CRCA module can be composed of a cascade of multiple Region Context Reassembly (RCA) modules to complete the upsampling process from the original scale to the target scale step by step. The number of RCA modules that need to be cascaded can be determined according to the target scale S.i For example, the original size is 4*4, and the target size after upsampling is 16*16, then S i is 4.

[0063] The specific calculation formula for the number of RCA modules that need to be cascaded is: N i =log2(S i ),i∈{2,...,μ max}. Among them, N i is the number of RCA modules required, and i is the level label of the encoded multi-channel feature map. For the large-scale upsampling process of the encoded multi-channel feature map, the Cascade Region Context Reassembly (CRCA) module is used to reduce the network's fitting difficulty during large-scale upsampling and improve the network's perception of text instance regions.

[0064] Combine Figure 2-1 , Figure 2-1 A schematic diagram of a dynamic perception text segmentation network structure based on text mask provided in an embodiment of the present invention is used as an example: after the encoding network HRNet (Encoder in the figure) extracts and encodes the original input image (i.e., printed text image and standard text image), it outputs feature maps x1, x2, x3, and x4. Then x2 and x3 can be sent to the CRCA module in the multi-level feature fusion module (Multi-Feature Fuse in the figure). At the same time, for the high-level feature map x4, the bidirectional attention mechanism DAM is used to reconstruct its features and then input it into the CRCA module. Then, x1 and the feature maps x2, x3, and x4 processed by the CRCA module are input into the Concat layer together. Finally, the result of the Concat layer is input into the nonlinear feature mapping module (Conv in the figure) composed of the convolution layer, the pooling layer, and the batch normalization layer to perform feature mapping on the combined features and complete the feature aggregation function. The rough segmentation result output by the multi-level feature fusion module is denoted as F coarse (The figure uses x coarse Represents), the semantic feature vector is F semantics (The figure uses x semantics Representation), where the coarse segmentation result map can be understood as a rougher text image segmentation map, and the semantic feature vector can be understood as the feature representation of each pixel, for example, whether the pixel is text or background.

[0065] Then, the output x of the multi-level feature fusion module can be coarse and x semanticsAs the input of the decoding network (Decoder in the figure), through the classic semantic segmentation decoding network, such as the target context representation network OCR (Object Contextual Representations), DA (Dual Attention), etc., combined with the semantic feature vector F semantics , for the coarse segmentation result image F coarse Perform feature enhancement to generate the multi-channel text segmentation feature map (which can also be understood as the final feature representation y for each pixel) i , in the figure, x predition The final feature expression of each pixel is as follows:

[0066]

[0067]

[0068]

[0069] Among them, y i is the final feature representation of pixel i, K is the total number of target regions, obtained from the feature map output by the encoding network (HRNet), and the image I is divided into k soft target regions, f k is the image feature of the kth target area, m k i is the pixel p in the kth target area i The normalization degree of x is obtained by normalizing each target area using the Softmax function. i is pixel p i The characteristic representation of ω ik is the relationship between the i-th pixel and the k-th target area, κ(x,f)=φ(x) T ψ(f) is the unnormalized relational function, and δ(·), ρ(·), φ(·), and ψ(·) are transformation functions, calculated via 1×1Conv→BN→ReLU. It can be understood that the generated multi-channel text fine segmentation feature maps include those corresponding to printed text images and those corresponding to standard text images.

[0070] S102. Use an image refinement algorithm to extract text skeleton information from the multi-channel text fine segmentation feature map, assign different weights to each layer of pixels according to the order in which pixels are removed during the extraction process, and establish a text skeleton template; the text skeleton template includes a printed text skeleton template and a standard text skeleton template.

[0071] Furthermore, the ZS refinement algorithm can be used to scan the multi-channel text fine segmentation feature map pixel by pixel, and by performing arithmetic and logical operations on the eight neighborhoods of the target pixel, the target pixel whose operation results meet the preset conditions is removed to obtain the text skeleton of the multi-channel text fine segmentation feature map; then, different weights are assigned to each layer of pixels according to the order of removing pixels, and a text skeleton template is established based on these weights.

[0072] In this embodiment, when extracting text skeleton information using an image thinning algorithm, the algorithm is iterated repeatedly until no pixels in the image can be deleted, thereby obtaining the text skeleton of the multi-channel text fine segmentation feature map. Weights are assigned to each layer of pixels based on the order in which pixels are removed during text thinning. Pixels in the outer layers are less important to the text as a whole and therefore have smaller weights. Pixels closer to the skeleton receive larger weights, with the largest weight representing the text skeleton, indicating that the skeleton is the most important.

[0073] For example, the ZS thinning algorithm is used to extract text skeletons. The ZS thinning algorithm uses arithmetic logic operations to determine pixel deletion. The target image is scanned pixel by pixel, and arithmetic logic operations are performed on the eight neighborhoods of the contour point pixels. The result of the operation determines whether the pixel should be deleted. The eight-neighborhood structure is shown below:

[0074] <![CDATA[P9]]> <![CDATA[P2]]> <![CDATA[P3]]> <![CDATA[P8]]> <![CDATA[P1]]> <![CDATA[P4]]> <![CDATA[P7]]> <![CDATA[P6]]> <![CDATA[P5]]>

[0075] In the eight-neighborhood, the pixel value of the foreground point is 1, and the pixel value of the background point is 0. Image parallel refinement is divided into two parts:

[0076] (1) For any pixel value p1 with 1, if its eight neighbors satisfy the following four conditions:

[0077] a.2≤N(p1)≤6,

[0078] bS(p1)=1,

[0079] c.p2×p4×p6=0,

[0080] d.p4×p6×p8=0;

[0081] Then p1 is marked as the point to be deleted and will be deleted after the current scan is completed. Where N(p1) represents the number of non-zero points in the eight-neighborhood, and S(p1) represents the number of times the pixel value of p1, p2, p3, L, and p9 changes from 0 to 1 in the clockwise direction;

[0082] (2) If p1 satisfies conditions a and b and the following conditions:

[0083] e.p2×p4×p8=0,

[0084] f.p2×p6×p8=0;

[0085] Then p1 is also marked as a point to be deleted and deleted after this scan is completed; the algorithm is iterated repeatedly until there are no pixels that can be deleted in the image, that is, the text skeleton is obtained and the image refinement is completed.

[0086] Then, different weights are assigned to each layer of pixels according to the order of removing pixels, and a text skeleton template is established based on these weights: Let T i is the set of pixels removed each time. According to the importance of the removed pixels, the weight W(i) of each layer of pixels is as follows:

[0087]

[0088] Where n represents the number of refinements, then the skeleton template S(i) of the text can be expressed as:

[0089] S(i)=1×T1+2×T2+L+W(n)×T n

[0090] The text skeleton template fully represents the visual importance of the text stroke pixels, so that the text printing quality evaluation combines the objective evaluation criteria of text quality with the subjective perception of the human eye.

[0091] S104 , using a structural similarity image quality evaluation algorithm to perform text printing quality detection on the standard text skeleton template and the printed text skeleton template, and obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template.

[0092] Specifically, when performing text printing quality inspection on the standard text skeleton template and the printed text skeleton template, a window of a specified size can be designed, and the window can be moved in pixels each time to calculate the structural similarity value of the image in each window to obtain multiple local structural similarity values; then, the multiple local structural similarity values are averaged to obtain the average structural similarity value between the printed text skeleton template and the standard text skeleton template.

[0093] In this embodiment, when calculating the structural similarity index of two images, a local window is designed, generally a small block of N×N, and the structural similarity value SSIM of the signal within the window is calculated. The window is moved in units of pixels each time until the local structural similarity value of each position in the entire image is calculated. The average of all local structural similarity values is the average structural similarity value MSSIM of the two images. Figure 2-2 SSIM can measure the similarity between two images from three aspects: brightness, contrast, and structure (assuming that the printed text skeleton template is image x and the standard text skeleton template is image y).

[0094] Brightness contrast function:

[0095] Contrast contrast function:

[0096] Structural comparison function:

[0097] Structural similarity index SSIM: SSIM(x,y)=[I(x,y)] α [c(x,y)] β [s(x,y)] γ ;

[0098] Average structural similarity index MSSIM:

[0099] Here, α, β, and γ are generally set to 1, and Simplifying, we get:

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] Where N=8, H and W are the height and width of the window respectively, μ x , μ y represents the mean of image x and image y, σ x and σ y Represents the variance of image x and image y, σ xy Represents the covariance of image x and image y, C1=(K1L) 2 , C2=(K2L) 2 represents the correlation coefficient, L is the grayscale level of the image, take L = 255, K1, K2 are constants and K1, K2 = 1, take K1 = 0.01, K2 = 0.03.

[0107] S106: Determine the text printing quality based on the average structural similarity value.

[0108] The embodiment of the present invention provides a text printing quality evaluation method based on text mask structural similarity: first, a dynamic perceptual text segmentation network based on text mask is constructed to process a printed text image and a standard text image to obtain a multi-channel text fine segmentation feature map, wherein the printed text image and the standard text image have the same image content, the dynamic perceptual text segmentation network based on text mask is pre-trained by a text image dataset with real labels, the real labels carry the characters and semantic mask information in the text image, and the multi-channel text fine segmentation feature map carries the semantic mask information; secondly, an image refinement algorithm is used to extract text skeleton information from the aforementioned multi-channel text fine segmentation feature map, and different weights are assigned to each layer of pixels according to the order in which pixels are removed during the extraction process, and a text skeleton template is established, wherein the text skeleton template includes a printed text skeleton template and a standard text skeleton template; then, a structural similarity image quality evaluation algorithm is used to perform text printing quality detection on the standard text skeleton template and the printed text skeleton template to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template; finally, the text printing quality is determined based on the average structural similarity value. The dynamic perception text segmentation network based on text mask constructed by the present invention outputs a multi-channel text fine segmentation feature map with text mask information, that is, the pixels in the text area have mask information, so that the output result can avoid the defects of incomplete text strokes, broken strokes, unclear edges, etc. caused by the complex background in the ordinary binarization process in traditional technology, thereby improving the detection accuracy; and through the constructed dynamic perception text segmentation network based on text mask, it can solve the problems of text instances of arbitrary structures that may exist in the input image and the large difference in pixel texture between different text instances, thereby improving the detection range and detection accuracy. The present invention constructs a multi-level feature fusion module, uses a parameter-free calculation method to enhance the text feature representation, performs feature fusion on the input multi-level features, improves the network's modeling ability for multi-shape text of the input data, and effectively improves the performance of the network.

[0109] The following describes the pre-training of the dynamic perceptual text segmentation network based on text masks. Figure 3 , Figure 3 This is a diagram of the training process of the dynamic perceptual text segmentation network based on text mask, as shown in Figure 3 As shown, the training process includes:

[0110] S300: Acquire a training text image sample, where the training text image sample is a text image with a true label, and the true label carries character and semantic mask information in the training text image sample.

[0111] In this embodiment, the training set and test set can be obtained first:

[0112] For example, M image samples with real labels can be randomly selected from the scene text dataset to form a training set (i.e., a set of training text image samples). At the same time, N image samples with real labels can be selected to form a test set. The test set images are standard text images, and these N images are printed to produce N corresponding printed text images (which can be used to test whether the trained network is qualified). Each real label contains character and semantic mask information. Usually, M ≥ 5000 and N ≥ 500. Here, M = 5000 and N = 500 can be selected.

[0113] S302: Process the training text image samples to obtain predicted labels.

[0114] S304: Calculate the loss value between the predicted label and its corresponding true label.

[0115] S306. Based on the loss value, iteratively train the text mask-based dynamic perception text segmentation network by adopting the gradient descent method.

[0116] In this embodiment, let the number of iterations be t, the maximum number of iterations be T, and generally the value of T can be T≥62500, and let t=0; randomly select M image samples with real labels from the training set as the input of the dynamic perceptual text segmentation network based on text mask for training, and obtain M predicted labels, and then calculate the loss value L of the M predicted labels and their corresponding real labels.

[0117] The calculation formula of the loss value L is as follows:

[0118] L=L coarse +αL refine +βL reg ;

[0119] L coarse =CrossEntropy(F coarse ,F label );

[0120] L refine =CrossEntropy(F refine ,F label );

[0121] L reg =CrossEntropy(F reg ,F label ).

[0122] Among them, L coarse is the loss value of the text coarse segmentation map, L refine is the loss value of the multi-channel text fine segmentation feature map, L reg is the rough segmentation graph F from the textcoarse The character recognition loss value is obtained by cutting the corresponding sub-image area and feeding it into the character recognition network; CrossEntropy(·) is the cross entropy loss function, F label is the true label value, F refine is the multi-channel text fine segmentation feature map, F reg is the result value of the character recognition network, α and β are preset values, which can be set to α = 0.5 and β = 0.1 in this embodiment. Then, the gradient descent method is used to update the parameters of each layer of the dynamic perceptual text segmentation network based on text masks by the loss value L. Until t>=T, the trained dynamic perceptual text segmentation network based on text masks is obtained. Otherwise, t=t+1 is set and training continues.

[0123] In this embodiment, an additional character recognition network, such as LeNet, is added to form a loss function to optimize the performance of the network; the character information contained in the training set and the character information predicted by the character recognition network can establish a loss, thereby improving the performance of the network.

[0124] The text mask-based dynamic perception text segmentation network model trained in this embodiment can be used to process situations such as the presence of text instances with arbitrary structures in the input image and large differences in pixel texture between different text instances. It can also evaluate text printing defects such as broken strokes, incomplete strokes, and white spots that appear in text during printing production, thereby improving and perfecting the text printing quality evaluation.

[0125] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0126] Memory 403, used for storing computer programs;

[0127] The processor 401 is configured to execute the program stored in the memory 403 by performing the following steps:

[0128] A dynamic perceptual text segmentation network based on text masks is constructed to process printed text images and standard text images to obtain a multi-channel text fine segmentation feature map; the printed text image and the standard text image have the same image content, the dynamic perceptual text segmentation network based on text masks is pre-trained using a text image dataset with real labels, and the multi-channel text fine segmentation feature map carries semantic mask information;

[0129] Using an image thinning algorithm to extract text skeleton information from the multi-channel text fine segmentation feature map, assigning different weights to each layer of pixels according to the order in which pixels are removed during the extraction process, and establishing a text skeleton template; the text skeleton template includes a printed text skeleton template and a standard text skeleton template;

[0130] Using a structural similarity image quality evaluation algorithm, the standard text skeleton template and the printed text skeleton template are subjected to text printing quality detection to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template;

[0131] The text printing quality is determined according to the average structural similarity value.

[0132] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0133] The communication interface is used for communication between the above electronic device and other devices.

[0134] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0135] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0136] The method provided in the embodiments of the present invention can be applied to electronic devices. Specifically, the electronic devices can be desktop computers, portable computers, smart mobile terminals, servers, etc. This is not limited here; any electronic device that can implement the present invention falls within the scope of protection of the present invention.

[0137] It should be noted that the device, electronic device and storage medium of the embodiments of the present invention are respectively the device, electronic device and storage medium for evaluating text printing quality using the above-mentioned text mask structure similarity. All embodiments of the above-mentioned text printing quality evaluation using the above-mentioned text mask structure similarity are applicable to the device, electronic device and storage medium, and can achieve the same or similar beneficial effects.

[0138] By using the terminal device provided by the embodiment of the present invention, proper nouns and / or fixed phrases can be displayed for user selection, thereby reducing user input time and improving user experience.

[0139] The terminal device exists in various forms, including but not limited to:

[0140] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0141] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0142] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0143] (4) Other electronic devices with data interaction functions.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0145] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0146] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and implement other changes to the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0147] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices (equipment), or computer program products. Therefore, the application can adopt the form of complete hardware embodiment, complete software embodiment, or the embodiment in combination with software and hardware, which are all collectively referred to as "module" or "system" herein. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The computer program is stored / distributed in a suitable medium, provided together with other hardware or as a part of hardware, or other distribution forms can be adopted, such as by the Internet or other wired or wireless telecommunication systems.

[0148] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (devices) and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0151] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A text printing quality evaluation method based on text mask structure similarity, characterized in that: The text printing quality evaluation method based on text mask structure similarity includes: A dynamic perceptual text segmentation network based on text masks is constructed to process printed text images and standard text images to obtain a multi-channel text fine segmentation feature map; the printed text image and the standard text image have the same image content, the dynamic perceptual text segmentation network based on text masks is pre-trained using a text image dataset with real labels, and the multi-channel text fine segmentation feature map carries semantic mask information; Using an image thinning algorithm to extract text skeleton information from the multi-channel text fine segmentation feature map, assigning different weights to each layer of pixels according to the order in which pixels are removed during the extraction process, and establishing a text skeleton template; the text skeleton template includes a printed text skeleton template and a standard text skeleton template; Using a structural similarity image quality evaluation algorithm, the standard text skeleton template and the printed text skeleton template are subjected to text printing quality detection to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template; determining the printing quality of the text according to the average structural similarity value; The method of using an image thinning algorithm to extract text skeleton information from the multi-channel text fine segmentation feature map, assigning different weights to each layer of pixels according to the order of removing pixels during the extraction process, and establishing a text skeleton template includes: The ZS refinement algorithm is used to scan the multi-channel text fine segmentation feature map pixel by pixel. By performing arithmetic and logical operations on the eight neighborhoods of the target pixel, the target pixels whose operation results meet the preset conditions are removed to obtain the text skeleton of the multi-channel text fine segmentation feature map; different weights are assigned to each layer of pixels according to the order of removing pixels, and a text skeleton template is established based on the weights.

2. The text printing quality evaluation method based on text mask structure similarity according to claim 1, characterized in that: The dynamic perceptual text segmentation network based on text mask includes an encoding network, a multi-level feature fusion module, and a decoding network; The dynamic perceptual text segmentation network based on text mask is constructed to process printed text images and standard text images to obtain multi-channel text fine segmentation feature maps, including: Performing feature extraction and feature encoding on the printed text image and the standard text image through the encoding network to obtain an encoded multi-channel feature map; Through the multi-level feature fusion module, the encoded multi-channel feature map is processed including feature reconstruction, splicing and feature mapping to complete the feature aggregation function and obtain the rough segmentation result map F coarse and semantic feature vector F semantics ; Through the decoding network, combined with the semantic feature vector F semantics For the coarse segmentation result image F coarse Feature enhancement is performed to generate the multi-channel text fine segmentation feature map.

3. The text printing quality evaluation method based on text mask structure similarity according to claim 2, characterized in that: The multi-level feature fusion module includes a feature enhancement module, a Concat layer, and a nonlinear feature mapping module; The multi-level feature fusion module performs feature reconstruction, splicing and feature mapping on the encoded multi-channel feature map to complete the feature aggregation function and obtain a rough segmentation result map F. coarse and semantic feature vector F semantics ,include: Performing feature reconstruction on the encoded multi-channel feature map through the feature enhancement module, and inputting the reconstructed feature map into the Concat layer; Using the Concat layer, the reconstructed feature maps are spliced by channel to obtain a spliced feature map; The nonlinear feature mapping module is used to perform feature mapping on the spliced feature map to complete the feature aggregation function and obtain the rough segmentation result map F. coarse and semantic feature vector F semantics .

4. The text printing quality evaluation method based on text mask structure similarity according to claim 1, characterized in that: The method of using a structural similarity image quality evaluation algorithm to perform text printing quality detection on the standard text skeleton template and the printed text skeleton template to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template includes: Design a window of a specified size, move the window in pixels each time, calculate the structural similarity value of the image in each window, and obtain multiple local structural similarity values; The multiple local structural similarity values are averaged to obtain an average structural similarity value between the printed text skeleton template and the standard text skeleton template.

5. The text printing quality evaluation method based on text mask structure similarity according to claim 2, characterized in that: Pre-training the text mask-based dynamic-aware text segmentation network includes: Acquire a training text image sample, where the training text image sample is a text image with a true label, and the true label carries character and semantic mask information in the training text image sample; The text mask-based dynamic perceptual text segmentation network is trained using the training text image samples.

6. The text printing quality evaluation method based on text mask structure similarity according to claim 5, characterized in that: The step of training the text mask-based dynamic perceptual text segmentation network using the training text image samples comprises: Processing the training text image sample to obtain a predicted label; Calculate the loss value of the predicted label and its corresponding true label; Based on the loss value, the text mask-based dynamic perceptual text segmentation network is iteratively trained by adopting a gradient descent method.

7. The text printing quality evaluation method based on text mask structure similarity according to claim 6, characterized in that: Calculating the loss value of the predicted label and its corresponding true label includes: The loss value is calculated by the following formula: L=L coarse +αL refine +βL reg ; L coarse =CrossEntropy(F coarse ,F label ); L refine =CrossEntropy(F refine ,F label ); L reg =CrossEntropy(F reg ,F label ); Among them, L coarse is the loss value of the text coarse segmentation map, L refine is the loss value of the multi-channel text fine segmentation feature map, L reg For the rough segmentation graph F from the text coarse The character recognition loss value is obtained by cutting the corresponding sub-image area and feeding it into the character recognition network; CrossEntropy(·) is the cross entropy loss function, F label is the true label value, F refine is the multi-channel text fine segmentation feature map, F reg is the result value of the character recognition network, and α and β are preset values.

8. The text printing quality evaluation method based on text mask structure similarity according to claim 2, characterized in that: The decoding network is the target context representation network OCR.

Images