Character merging method, device, electronic device and storage medium

Abstract

Embodiments of the present invention provide a method, device, electronic device and storage medium for character merging. The method for character merging includes: extracting single character regions from an image to be processed; obtaining image attribute features and coordinate positions of the single character regions for each single character region; merging single character regions belonging to the same text region based on the image attribute features and coordinate positions of the single character regions. After the embodiment of the present invention merges the single character regions based on the image attribute features and coordinate positions of the single character regions, the accuracy of the obtained text region is higher.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a character merging method, device, electronic equipment and storage medium. Background Art

[0002] In the process of image processing, it is often necessary to extract text areas in images. For example, in the process of page development, in order to enrich the display effect of the page, a large number of images are often set in the page. For these images, they need to be converted into code files so that the terminal can render the corresponding images in the page by running these code files. If there are text areas in these images, when converting the images into code files, these text areas need to be extracted and converted separately.

[0003] In the prior art, text regions are generally extracted from images using text detection models. However, the accuracy of text regions extracted by text detection models is low. For example, the edges of the extracted text regions contain many image regions that are not text, or text at the edges is not extracted into the text region, etc. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a character merging method, device, electronic device and storage medium to improve the accuracy of text areas. The specific technical solution is as follows:

[0005] In the first aspect of the implementation of the present invention, a character merging method is first provided, including: extracting single-character areas from an image to be processed; for each single-character area, respectively obtaining the image attribute characteristics and coordinate positions of the single-character area; and merging the single-character areas belonging to the same text area based on the image attribute characteristics and coordinate positions of the single-character area.

[0006] Optionally, the single-character areas belonging to the same text area are merged based on the image attribute characteristics and coordinate positions of the single-character areas, including: taking a single-character area as a region to be merged; repeating the following steps: for every two regions to be merged, judging whether the two regions to be merged belong to the same text area based on the image attribute characteristics and coordinate positions of the two regions to be merged; when it is judged for the first time that the two regions to be merged belong to the same text area, merging the two regions to be merged into one region to be merged, and obtaining the image attribute characteristics and coordinate positions of the merged region to be merged; when the merging is no longer to be continued, taking the current region to be merged as a text area.

[0007] Optionally, the image attribute features include color values; and judging whether the two areas to be merged belong to the same text area based on the image attribute features and coordinate positions of the two areas to be merged includes: judging whether the two areas to be merged overlap based on the coordinate positions of the two areas to be merged; when there is overlap, judging that the two areas to be merged belong to the same text area; when there is no overlap, judging whether the two areas to be merged are located in the same row or column based on the coordinate positions of the two areas to be merged; when they are located in the same row or column, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distance between the two areas to be merged is similar based on the coordinate positions of the two areas to be merged; when the colors and the distance are similar, judging that the two areas to be merged belong to the same text area.

[0008] Optionally, before merging single-character areas belonging to the same text area based on the image attribute features and coordinate positions of the single-character areas, it also includes: for each single-character area, identifying whether the single-character area is a first-category punctuation area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged; when the colors and the distances are similar, judging that the two areas to be merged belong to the same text area, including: judging whether there is at least one first-category punctuation area in the two areas to be merged; if not, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged; when the colors and the distances are similar, judging that the two areas to be merged belong to the same text area; if so, judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged; when the distances are similar, judging that the two areas to be merged belong to the same text area.

[0009] Optionally, the image attribute features also include width and height; for any single-character area, identifying whether the any single-character area is a first-category punctuation area, including: extracting a control text area from the image to be processed, and obtaining the coordinate position and height of the control text area; based on the coordinate position of the any single-character area and the coordinate position of the control text area, selecting a control text area that overlaps with the any single-character area, and obtaining the maximum height of the selected control text area; if the width and height of the any single-character area are both less than a first preset percentage of the maximum height, determining that the any single-character area is a first-category punctuation area.

[0010] Optionally, before merging single-character areas belonging to the same text area based on the image attribute features and coordinate positions of the single-character areas, it also includes: for each single-character area, separately identifying whether the single-character area is a second-category punctuation area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, including: judging whether there is at least one second-category punctuation area in the two areas to be merged; if not, selecting a preset first color threshold, and judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the first color threshold; if so, selecting a preset second color threshold, and judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the second color threshold; the first color threshold is less than the second color threshold.

[0011] Optionally, the image attribute features also include width and height; for any single-character area, identifying whether the any single-character area is a second-category punctuation area, including: extracting a control text area from the image to be processed, and obtaining the coordinate position and height of the control text area; based on the coordinate position of the any single-character area and the coordinate position of the control text area, selecting a control text area that overlaps with the any single-character area, and obtaining the maximum height of the selected control text area; if the width or height of the any single-character area is less than a second preset percentage of the maximum height, determining that the any single-character area is a second-category punctuation area.

[0012] Optionally, the color value includes a foreground color value and a background color value; for any single-character area, obtaining the color value of any single-character area, including: for each pixel in the any single-character area, obtaining the RGB color value of the pixel; clustering all pixels according to the RGB color value of the pixel to obtain two clusters; for each cluster, respectively obtaining the RGB color value corresponding to the cluster; converting the RGB color value corresponding to the cluster into an HSV color value, using the HSV color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and using the HSV color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, including: judging whether the colors of the two areas to be merged are similar based on the hue H component in the background color value and the hue H component in the foreground color value of the two areas to be merged.

[0013] Optionally, the color value includes a foreground color value and a background color value; for any single-character area, obtaining the color value of any single-character area, including: for each pixel in the any single-character area, obtaining the RGB color value of the pixel; clustering all pixels according to the RGB color value of the pixel to obtain two clusters; for each cluster, respectively obtaining the RGB color value corresponding to the cluster; using the RGB color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and using the RGB color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, including: judging whether the colors of the two areas to be merged are similar based on the background color value and foreground color value of the two areas to be merged.

[0014] In the second aspect of the implementation of the present invention, there is also provided a method comprising: an extraction module for extracting single-character areas from an image to be processed; an acquisition module for acquiring, for each single-character area, the image attribute characteristics and coordinate positions of the single-character area; and a merging module for merging single-character areas belonging to the same text area based on the image attribute characteristics and coordinate positions of the single-character areas.

[0015] Optionally, the merging module includes: an initialization submodule, used to treat a single character area as an area to be merged; an area merging submodule, used to repeatedly execute the following steps: for every two areas to be merged, judging whether the two areas to be merged belong to the same text area based on the image attribute characteristics and coordinate positions of the two areas to be merged; when it is judged for the first time that the two areas to be merged belong to the same text area, merging the two areas to be merged into one area to be merged, and obtaining the image attribute characteristics and coordinate positions of the merged area to be merged; an area determination submodule, used to treat the current area to be merged as a text area when the merging is no longer to be continued.

[0016] Optionally, the image attribute features include color values; the region merging submodule includes: a first judgment unit, used to judge whether the two regions to be merged overlap based on the coordinate positions of the two regions to be merged; when the first judgment unit judges that there is an overlap, judge that the two regions to be merged belong to the same text area; a second judgment unit, used to judge whether the two regions to be merged are located in the same row or column based on the coordinate positions of the two regions to be merged when the first judgment unit judges that there is no overlap; a third judgment unit, used to judge whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged when the second judgment unit judges that they are located in the same row or column, and judge whether the distance between the two regions to be merged is similar based on the coordinate positions of the two regions to be merged; when the colors and the distances are similar, judge that the two regions to be merged belong to the same text area.

[0017] Optionally, the device also includes: a first recognition module, used to identify, for each single-character area, whether the single-character area is a first-class punctuation area before the merging module merges the single-character areas belonging to the same text area based on the image attribute characteristics and coordinate positions of the single-character areas; the third judgment unit is specifically used to judge whether there is at least one first-class punctuation area in the two areas to be merged; if not, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged, and when the colors and the distances are similar, judging that the two areas to be merged belong to the same text area; if so, judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged, and when the distances are similar, judging that the two areas to be merged belong to the same text area.

[0018] Optionally, the image attribute features also include width and height; the first recognition module includes: a first extraction submodule, used to extract a control text area from the image to be processed, and obtain the coordinate position and height of the control text area; a first acquisition submodule, used to select a control text area that overlaps with any single character area based on the coordinate position of any single character area and the coordinate position of the control text area, and obtain the maximum height of the selected control text area; a first comparison submodule, used to determine that any single character area is a first-class punctuation area if the width and height of any single character area are both less than a first preset percentage of the maximum height.

[0019] Optionally, the device also includes: a second recognition module, used to identify, for each single-character area, whether the single-character area is a second-category punctuation area before the merging module merges the single-character areas belonging to the same text area based on the image attribute characteristics and coordinate positions of the single-character areas; the third judgment unit is specifically used to judge whether there is at least one second-category punctuation area in the two areas to be merged when judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged; if not, a preset first color threshold is selected to judge whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the first color threshold; if so, a preset second color threshold is selected to judge whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the second color threshold; the first color threshold is less than the second color threshold.

[0020] Optionally, the image attribute features also include width and height; the second recognition module includes: a second extraction submodule, used to extract a control text area from the image to be processed, and obtain the coordinate position and height of the control text area; a second acquisition submodule, used to select a control text area that overlaps with any single character area based on the coordinate position of any single character area and the coordinate position of the control text area, and obtain the maximum height of the selected control text area; a second comparison submodule, used to determine that any single character area is a second type of punctuation area if the width or height of any single character area is less than a second preset percentage of the maximum height.

[0021] Optionally, the color value includes a foreground color value and a background color value; the acquisition module includes: a first color acquisition submodule, used to acquire the RGB color value of the pixel for each pixel in any single-character area; a clustering submodule, used to cluster all pixels according to the RGB color value of the pixel to obtain two clusters; a second color acquisition submodule, used to acquire the RGB color value corresponding to the cluster for each cluster; a first color determination submodule, used to convert the RGB color value corresponding to the cluster into an HSV color value, and use the HSV color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and use the HSV color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; the third judgment unit is specifically used to judge whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged.

[0022] Optionally, the color value includes a foreground color value and a background color value; the acquisition module includes: a first color acquisition submodule, used to acquire the RGB color value of the pixel for each pixel in any single-character area; a clustering submodule, used to cluster all pixels according to the RGB color value of the pixel to obtain two clusters; a second color acquisition submodule, used to acquire the RGB color value corresponding to the cluster for each cluster; a second color determination submodule, used to use the RGB color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and use the RGB color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; the third judgment unit is specifically used to judge whether the colors of the two areas to be merged are similar based on the background color value and foreground color value of the two areas to be merged when judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged.

[0023] In another aspect of the implementation of the present invention, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement any of the above-mentioned character merging methods when executing the program stored in the memory.

[0024] In another aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed, the computer implements any of the above-mentioned character merging methods.

[0025] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to implement any of the above-mentioned character merging methods.

[0026] The character merging method, device, electronic device and storage medium provided by the embodiment of the present invention extract single character regions from the image to be processed; for each single character region, the image attribute characteristics and coordinate position of the single character region are respectively obtained; and the single character regions belonging to the same text region are merged according to the image attribute characteristics and coordinate position of the single character region. Since the extracted single character region will not have the situation that the edge contains a large number of image regions that are not texts, or the text on the edge is not extracted into the text region, the accuracy of the text region obtained after the single character regions are merged based on the image attribute characteristics and coordinate position of the single character regions is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0028] Figure 1 The figure is a flowchart of a character merging method in an embodiment of the present invention.

[0029] Figure 2 The figure is a flowchart of another character merging method in an embodiment of the present invention.

[0030] Figure 3 The following is a flowchart of another character merging method in an embodiment of the present invention.

[0031] Figure 4 This is an effect diagram of extracting a text area in an embodiment of the present invention.

[0032] Figure 5 This is an effect diagram of extracting a single character area in an embodiment of the present invention.

[0033] Figure 6 The figure is a diagram showing the effect of character merging in an embodiment of the present invention.

[0034] Figure 7 This is another effect diagram of extracting a text area in an embodiment of the present invention.

[0035] Figure 8 This is another effect diagram of extracting a single character area in an embodiment of the present invention.

[0036] Fig. 9 This is another effect diagram after character merging in an embodiment of the present invention.

[0037] Fig.10 The figure is a structural block diagram of a character merging device in an embodiment of the present invention.

[0038] Fig.11 It is a structural block diagram of another character merging device in an embodiment of the present invention.

[0039] Fig.12 The figure is a structural block diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0041] In view of the problem that the text area extracted from the image by the text detection model in the prior art has low accuracy, the embodiment of the present invention proposes to first extract the single character area from the image, and then merge the single character area according to the image attribute characteristics and coordinate positions of the single character area to obtain the text area. Since the accuracy of extracting the single character area is higher than that of extracting the entire text area, the text area obtained by merging the single character areas is higher than that of the text area directly extracted.

[0042] The embodiments of the present invention can be applied to any electronic device with image processing capability. The electronic device can be a terminal or a server, wherein the terminal can be a smart phone, a tablet computer, a wearable device, a personal computer, etc.

[0043] Figure 1 The figure is a flowchart of a character merging method in an embodiment of the present invention.

[0044] like Figure 1 As shown, the character merging method may include the following steps:

[0045] Step 101, extracting a single character region from the image to be processed.

[0046] The image to be processed can be any image whose text area needs to be extracted. For example, for images contained in a page, these images need to be converted into code files, and the text area in the image needs to be extracted and converted separately, so the images contained in the page can be used as the image to be processed.

[0047] By performing image segmentation on the image to be processed, each single character region can be extracted from the image to be processed. The single character region may include words, letters, punctuation marks, numbers, and the like.

[0048] Step 102: for each single-character region, respectively obtain the image attribute features and coordinate positions of the single-character region.

[0049] The image attribute features of the single character region may include the color value, width, height, etc. of the single character region. The coordinate position of the single character region may include the abscissa and ordinate of the four vertices of the region.

[0050] Step 103: merging the single character regions belonging to the same text region according to the image attribute features and coordinate positions of the single character regions.

[0051] Based on the image attribute features and coordinate positions of the single-character regions, it is possible to identify whether different single-character regions belong to the same text region. By merging the single-character regions belonging to the same text region, the text region in the image to be processed can be obtained.

[0052] In the embodiment of the present invention, since the extracted single-character area will not have a situation where the edges contain a large number of image areas that are not texts, or the text on the edges is not extracted into the text area, the text area obtained after the single-character areas are merged based on the image attribute features and coordinate positions of the single-character areas will have higher accuracy.

[0053] Figure 2 The figure is a flowchart of another character merging method in an embodiment of the present invention.

[0054] like Figure 2 As shown, the character merging method may include the following steps:

[0055] Step 201, extracting a single character region from the image to be processed.

[0056] By performing character segmentation processing on the image to be processed, multiple single-character regions can be extracted from the image to be processed.

[0057] In an optional implementation, a single character area can be extracted from the image to be processed in the following manner: converting the image to be processed into a grayscale image; converting the grayscale image into a binary image using an edge detection algorithm; obtaining a character frame from the binary image using a projection method; and determining the single character area based on the character frame.

[0058] The image to be processed can be in RGB (Red, Green, Blue) format. The grayscale value of each pixel in the image to be processed can be determined by various methods such as floating-point arithmetic, integer method, shift method, average method, etc., and the grayscale value is used to replace the pixel value of the red, green and blue channels in the pixel point to obtain a grayscale image.

[0059] The edge detection algorithm can use algorithms such as Canny operator, Roberts operator, Sobel operator, Kirsh operator, etc. For example, the process of converting the grayscale image into a binary image based on the canny edge detection algorithm may include: 1. Gaussian filtering the grayscale image. Gaussian filtering can be achieved by weighting two one-dimensional Gaussian kernels twice, or by convolution of a two-dimensional Gaussian kernel once. Gaussian filtering the grayscale image can effectively filter out the high-frequency noise superimposed in the grayscale image. 2. Calculate the gradient amplitude and direction using the finite difference of the first-order partial derivative. The gradient of the grayscale value of the image can be approximated using the first-order finite difference, so as to obtain the first-order partial derivative matrix, gradient amplitude matrix and gradient direction matrix of the grayscale image in the x and y directions. 3. Perform non-maximum suppression processing according to the gradient amplitude. Non-maximum suppression in a popular sense refers to finding the local maximum value of the pixel point and setting the grayscale value corresponding to the non-maximum point to 0, so that a large part of the non-edge points can be eliminated. 4. Use a double threshold algorithm to detect and connect edges. The dual threshold algorithm sets a high threshold and a low threshold, and the high threshold can reduce the pseudo edges in the image, and the low threshold can close the edges in the image, so that the binary image can be obtained in the end.

[0060] The projection method is to analyze the distribution histogram of the pixels of the binary image to find the boundary points of adjacent character boxes and segment them accordingly. The projection method can include vertical projection and horizontal projection. For example, for vertical projection, each character box is determined based on the projection position with a projection value of 0 in the vertical projection image.

[0061] After the character frame is obtained, the character outline in the character frame may be detected using an outline detection algorithm, and then the single character area may be determined by the minimum circumscribed rectangular area of ​​the character outline in the character frame.

[0062] Step 202: for each single-character region, respectively obtain the image attribute features and coordinate positions of the single-character region.

[0063] Image attribute characteristics may include color value, width, height, and so on.

[0064] After extracting the single character region from the image to be processed, the coordinate position of each single character region can be obtained, and the coordinate position can include the coordinate positions of the four vertices of the single character region. The width and height of the single character region can be obtained according to the coordinate position of the single character region.

[0065] As for the image attribute feature of color value, the embodiment of the present invention adopts principal component color analysis on the single character area, and can extract two principal component colors of foreground color value and background color value.

[0066] In an optional implementation manner, for any single-character region, the process of obtaining the color value of the any single-character region may include the following steps A1 to A4:

[0067] Step A1, for each pixel in any single-character area, obtain the RGB color value of the pixel.

[0068] Step A2: clustering all pixels according to the RGB color values ​​of the pixels to obtain two clusters.

[0069] Each pixel in a single character region has a corresponding RGB color value. For a single character region, all pixels in the single character region are clustered according to the RGB color values ​​of the pixels in the single character region to obtain two clusters.

[0070] In an optional implementation, K-means clustering can be used. The clustering process may include: 1. Set the k value to 2, and randomly select 2 pixels from all the pixels in the single character area as the initial centroid. 2. For pixels other than the centroid, calculate the distance between each other pixel and the initial centroid (in implementation, the distance can be calculated based on the RGB color value), and assign each other pixel to the cluster with the nearest centroid. 3. Recalculate the centroids of the two clusters. 4. Repeat steps 2 and 3 until the cluster does not change, and use the last two clusters as the result.

[0071] Step A3: for each cluster, respectively obtain the RGB color value corresponding to the cluster.

[0072] For each cluster, the RGB color value of the centroid (i.e., pixel) of the cluster can be used as the RGB color value corresponding to the cluster. Alternatively, the average RGB color value of all pixels in the cluster can be calculated and the average value can be used as the RGB color value corresponding to the cluster.

[0073] Step A4, using the RGB color value corresponding to the cluster with more pixels as the background color value of any single character area, and using the RGB color value corresponding to the cluster with less pixels as the foreground color value of any single character area.

[0074] For a single character area, the character part occupies fewer pixels, and the background part occupies more pixels. Therefore, the RGB color value corresponding to the cluster with more pixels can be used as the background color value of the single character area, and the RGB color value corresponding to the cluster with fewer pixels can be used as the foreground color value of the single character area. Therefore, when comparing colors, the comparison can be based on the RGB color value.

[0075] In another optional implementation, for any single-character region, the process of obtaining the color value of the any single-character region may include the following steps B1 to B4:

[0076] Step B1, for each pixel in any single-character area, obtain the RGB color value of the pixel.

[0077] Step B2: clustering all pixels according to the RGB color values ​​of the pixels to obtain two clusters.

[0078] Step B3: for each cluster, respectively obtain the RGB color value corresponding to the cluster.

[0079] For the specific process of steps B1 to B3, refer to the relevant description of the above steps A1 to A3.

[0080] Step B4, converting the RGB color value corresponding to the cluster into an HSV color value, using the HSV color value corresponding to the cluster with a large number of pixels as the background color value of any single character area, and using the HSV color value corresponding to the cluster with a small number of pixels as the foreground color value of any single character area.

[0081] When converting the RGB color value to the HSV color value, the conversion formula of RGB to HSV can be used for conversion. The specific conversion process is not discussed in detail in the embodiment of the present invention.

[0082] In this implementation, the RGB color value is further converted into an HSV (Hue, Saturation, Value) color value. Since the H component of the HSV color value represents the hue, when comparing colors, the comparison can be based on the H component of the HSV color value, which is more convenient than comparing based on the RGB color value.

[0083] After the image attribute features and coordinate positions of each single character region are acquired, the single character regions belonging to the same text region are merged according to the image attribute features and coordinate positions of the single character regions. The merging process may include the following steps 203 to 206.

[0084] Step 203: taking a single character region as a region to be merged.

[0085] Step 204 : for each two regions to be merged, judging whether the two regions to be merged belong to the same text region according to the image attribute features and coordinate positions of the two regions to be merged.

[0086] All the regions to be merged are traversed, and judgment is performed on each two regions to be merged. In the embodiment of the present invention, the judgment of any two regions to be merged is taken as an example for explanation.

[0087] In an optional implementation manner, the process of determining whether the two regions to be merged belong to the same text region based on the image attribute features and coordinate positions of the two regions to be merged may include the following steps C1 to C5:

[0088] Step C1, based on the coordinate positions of the two regions to be merged, determine whether the two regions to be merged overlap. If yes, execute step C4; if no, execute step C2.

[0089] The coordinate positions of the regions to be merged are the horizontal coordinates and vertical coordinates of the four vertices. Based on the coordinate positions of the two regions to be merged, it is determined whether the two regions to be merged overlap. For example, it can be determined whether at least one horizontal coordinate of one of the regions to be merged is located between two different horizontal coordinates of the other region to be merged; if so, it is determined that the two regions to be merged overlap; if not, it is determined that the two regions to be merged do not overlap.

[0090] Step C2: when there is no overlap, based on the coordinate positions of the two regions to be merged, determine whether the two regions to be merged are located in the same row or the same column. If yes, execute step C3; if no, execute step C5.

[0091] Based on the coordinate positions of the two regions to be merged, determine whether the two regions to be merged are located in the same row or the same column. For example, obtain the overlapping height of the two regions to be merged in the y-axis direction. If the overlapping height is greater than the preset height threshold, it is determined that the two regions to be merged are located in the same row; otherwise, it is determined that the two regions to be merged are not located in the same row. Obtain the overlapping width of the two regions to be merged in the x-axis direction. If the overlapping width is greater than the preset width threshold, it is determined that the two regions to be merged are located in the same column; otherwise, it is determined that the two regions to be merged are not located in the same column.

[0092] For the specific values ​​of the height threshold and the width threshold, any applicable values ​​can be selected based on actual experience, and the embodiments of the present invention do not limit this. For example, the minimum height of the two areas to be merged can be selected, and a preset percentage of the minimum height can be used as the height threshold. The minimum width of the two areas to be merged can be selected, and a preset percentage of the minimum width can be used as the width threshold. The preset percentage can be 90%, 80%, etc., but in principle it cannot be too different from the minimum height or minimum width.

[0093] Step C3, when the two regions are located in the same row or the same column, based on the color values ​​of the two regions to be merged, determine whether the colors of the two regions to be merged are similar, and based on the coordinate positions of the two regions to be merged, determine whether the distances of the two regions to be merged are similar. When both the colors and the distances are similar, execute step C4; when at least one of the colors and the distances is not similar, execute step C5.

[0094] Based on the color values ​​of the two regions to be merged, it is determined whether the colors of the two regions to be merged are similar.

[0095] In an optional implementation, if the color value of the single character area is obtained according to the process of steps A1 to A4 above, the foreground color value and the background color value of the single character area are both RGB color values. Therefore, the process of judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged includes: judging whether the colors of the two areas to be merged are similar based on the background color value and the foreground color value of the two areas to be merged. In implementation, the distance between the foreground color values ​​of the two areas to be merged and the distance between the background color values ​​of the two areas to be merged can be calculated. If the distance between the foreground color values ​​of the two areas to be merged and the distance between the background color values ​​are both less than a preset RGB color threshold, it is determined that the colors of the two areas to be merged are similar; otherwise, it is determined that the colors of the two areas to be merged are not similar. For RGB color values, the distance can be obtained by calculating the three-dimensional Euclidean distance, Manhattan distance, Chebyshev distance, etc. of the RGB color values. For the specific value of the RGB color threshold, any applicable value can be selected based on actual experience, and the embodiment of the present invention does not limit this.

[0096] In another optional implementation, if the color value of the single character area is obtained according to the process of steps B1 to B4 above, the foreground color value and the background color value of the single character area are both HSV color values. Therefore, the process of judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged includes: judging whether the colors of the two areas to be merged are similar based on the H component in the background color value and the H component in the foreground color value of the two areas to be merged. In implementation, the distance between the H components in the background color values ​​of the two areas to be merged and the distance between the H components in the foreground color values ​​of the two areas to be merged can be calculated. If the distance between the H components in the foreground color values ​​of the two areas to be merged and the distance between the H components in the background color values ​​are both less than the preset H component color threshold, it is determined that the colors of the two areas to be merged are similar; otherwise, it is determined that the colors of the two areas to be merged are not similar. For the H component in the HSV color value, the distance can be obtained by calculating the difference of the H component. For the specific value of the H component color threshold, any applicable value can be selected based on actual experience, and the embodiment of the present invention does not limit this.

[0097] Based on the coordinate positions of the two regions to be merged, it is determined whether the distance between the two regions to be merged is close.

[0098] If two areas to be merged are located in the same row, then based on the coordinate positions of the two areas to be merged, it is determined whether the horizontal distances of the two areas to be merged are close. For example, the distances between the horizontal coordinates of one area to be merged and the horizontal coordinates of the other area to be merged are calculated respectively, and the calculated minimum distance is selected as the horizontal distance of the two areas to be merged. If the horizontal distance is less than a preset horizontal distance threshold, it can be determined that the horizontal distances of the two areas to be merged are close; otherwise, it is determined that the horizontal distances of the two areas to be merged are not close.

[0099] If two regions to be merged are located in the same column, then based on the coordinate positions of the two regions to be merged, it is determined whether the longitudinal distances of the two regions to be merged are close. For example, the distances between the longitudinal coordinates of one region to be merged and the longitudinal coordinates of the other region to be merged are calculated respectively, and the minimum distance calculated is selected as the longitudinal distance of the two regions to be merged. If the longitudinal distance is less than a preset longitudinal distance threshold, it can be determined that the longitudinal distances of the two regions to be merged are close; otherwise, it is determined that the longitudinal distances of the two regions to be merged are not close.

[0100] For the specific values ​​of the horizontal distance threshold and the vertical distance threshold, any applicable values ​​can be selected based on actual experience, and the embodiments of the present invention do not impose any restrictions on this. For example, considering that the single-character area is basically a rectangular area, there will not be a font size gap between normal typeset characters. Therefore, the maximum height of the two areas to be merged can be selected, and a preset percentage of the maximum height can be used as the horizontal distance threshold; or the maximum width of the two areas to be merged can be selected, and a preset percentage of the maximum width can be used as the horizontal distance threshold. The maximum height of the two areas to be merged can be selected, and a preset percentage of the maximum height can be used as the vertical distance threshold; or the maximum width of the two areas to be merged can be selected, and a preset percentage of the maximum width can be used as the vertical distance threshold. The preset percentage can be 90%, 80%, 70%, etc., but in principle it cannot be too different from the maximum height or maximum width.

[0101] It should be noted that, regarding the above-mentioned order of judging the color and the distance, the color may be judged first and then the distance, or the distance may be judged first and then the color. The embodiment of the present invention does not limit this.

[0102] Step C4: determining that the two regions to be merged belong to the same text region.

[0103] Step C5: determining that the two regions to be merged do not belong to the same text region.

[0104] Step 205 , when it is determined for the first time that the two regions to be merged belong to the same text region, the two regions to be merged are merged into one region to be merged, and the image attribute features and coordinate positions of the merged region to be merged are obtained, and the process returns to step 204 .

[0105] When it is determined for the first time in step 204 that two of the areas to be merged belong to the same text area, the two areas to be merged that belong to the same text area are merged into one area to be merged. At this time, it is no longer determined whether other areas to be merged belong to the same text area. For the merged area to be merged, the image attribute features and coordinate positions of the merged area to be merged are obtained, and then the process returns to step 204 and traverses again.

[0106] When obtaining the image attribute features and coordinate positions of the merged region to be merged, the coordinate position, width and height of the merged region to be merged can be obtained according to the coordinate positions of the two regions to be merged belonging to the same text region contained in the merged region to be merged. The color value of the merged region to be merged can be obtained according to the method for obtaining the color value described in step 202 above, and the above related description can be specifically referred to.

[0107] Step 206 : when it is not determined that the two regions to be merged belong to the same text region, it is determined not to continue merging, and the current region to be merged is regarded as a text region.

[0108] In step 204, if no two regions to be merged are determined to belong to the same text region, that is, any two regions to be merged do not belong to the same text region, it is determined that the merging is not continued, and the current region to be merged is regarded as a text region. Thus, the single-character regions belonging to the same text region are merged to obtain the text region in the image to be processed.

[0109] Figure 3 The following is a flowchart of another character merging method in an embodiment of the present invention.

[0110] like Figure 3 As shown, the character merging method may include the following steps:

[0111] Step 301: Obtain an image to be processed.

[0112] Step 302: extract a single character region from the image to be processed.

[0113] Step 303, obtaining the coordinate position, width, and height of each single character area.

[0114] Step 304: extracting a reference text region from the image to be processed.

[0115] Step 305, obtaining the coordinate position, width, and height of each comparison text area.

[0116] In an optional implementation, a text detection model can be used to extract a reference text region from the image to be processed. For example, a text detection model such as EAST (Efficient and Accuracy Scene Text), CTPN (Detecting Text in Natural Image with Connectionist Text Proposal Network), SegLink, etc. can be used to perform text detection on the image to be processed, and at least one reference text region can be extracted from the image to be processed.

[0117] After extracting the reference text area from the image to be processed, the coordinate position of each reference text area can be obtained, and the coordinate position can include the coordinate positions of the four vertices of the reference text area. The width and height of the reference text area can be obtained according to the coordinate position of the reference text area.

[0118] Step 306, obtaining the RGB color value of each single character area.

[0119] Step 307: cluster the pixels in each single-character region to obtain two clusters.

[0120] Step 308, obtaining the RGB color values ​​corresponding to the two clusters of each single-character region.

[0121] Step 309, converting the RGB color value into the HSV color value to obtain the foreground color value and the background color value of each single character area.

[0122] The HSV color value corresponding to the cluster with a large number of pixels is used as the background color value of any single-character area, and the HSV color value corresponding to the cluster with a small number of pixels is used as the foreground color value of any single-character area.

[0123] Step 310: Select a comparison text area that overlaps with the single character area, and obtain the maximum height of the selected comparison text area.

[0124] Step 311: If the width and height of the single-character region are both smaller than a first preset percentage of the maximum height, the single-character region is determined to be a first-category punctuation mark region.

[0125] Step 312: If the width or height of the single-character region is less than a second preset percentage of the maximum height, the single-character region is determined to be a second-category punctuation mark region.

[0126] In an embodiment of the present invention, before merging the single-character regions belonging to the same text region based on the image attribute features and coordinate positions of the single-character regions, it also includes: for each single-character region, identifying whether the single-character region is a first-category punctuation region. For any single-character region, the process of identifying whether the any single-character region is a first-category punctuation region includes: extracting a reference text region from the image to be processed, and obtaining the coordinate position and height of the reference text region; based on the coordinate position of the any single-character region and the coordinate position of the reference text region, selecting a reference text region that overlaps with the any single-character region, and obtaining the maximum height of the selected reference text region; if the width and height of the any single-character region are both less than a first preset percentage of the maximum height, determining that the any single-character region is a first-category punctuation region.

[0127] Before merging the single-character regions belonging to the same text region according to the image attribute features and coordinate positions of the single-character regions, the method further includes: for each single-character region, identifying whether the single-character region is a second-category punctuation region. The process of identifying whether any single-character region is a second-category punctuation region for any single-character region includes: extracting a reference text region from the image to be processed, and obtaining the coordinate position and height of the reference text region; based on the coordinate position of any single-character region and the coordinate position of the reference text region, selecting a reference text region that overlaps with any single-character region, and obtaining the maximum height of the selected reference text region; if the width or height of any single-character region is less than a second preset percentage of the maximum height, determining that any single-character region is a second-category punctuation region.

[0128] Among them, the first preset percentage is less than the second preset percentage. For the specific values ​​of the first preset percentage and the second preset percentage, any applicable values ​​can be selected according to actual experience. For example, the first category of punctuation marks is small punctuation marks (small punctuation marks include commas, periods, colons, etc.), and the second category of punctuation marks is normal punctuation marks (normal punctuation marks include parentheses, angle brackets, etc.). The first preset percentage is set according to the empirical value of the ratio between small punctuation marks and normal text, such as setting the first preset percentage to 15%, 20%, 25%, etc. The second preset percentage is set according to the empirical value of the ratio between normal punctuation marks and normal text, such as setting the second preset percentage to 30%, 35%, etc.

[0129] Step 313, obtain the image attribute features and coordinate position of the single character area, and take the single character area as a region to be merged. The image attribute features may include width, height, color value, and whether it is a punctuation mark (specifically, which type of punctuation mark).

[0130] Step 314, traverse all the areas to be merged.

[0131] During the traversal process, the following processing is performed for every two areas to be merged.

[0132] Step 315: Check whether the two regions to be merged overlap. If yes, execute step 320; if no, execute step 316.

[0133] Step 316: Check whether the two regions to be merged are located in the same row or the same column. If yes, execute step 317; if no, execute step 321.

[0134] Step 317: Whether there is at least one first-category punctuation mark region in the two regions to be merged. If yes, execute step 319; if no, execute step 318.

[0135] Step 318: Check whether the colors of the two regions to be merged are similar. If yes, go to step 319; if no, go to step 321.

[0136] In an optional implementation, determining whether the colors of two regions to be merged are similar may include: determining whether there is at least one second-category punctuation mark region in the two regions to be merged; if not, selecting a preset first color threshold, and determining whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged and the first color threshold; if yes, selecting a preset second color threshold, and determining whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged and the second color threshold. The first color threshold is smaller than the second color threshold.

[0137] In the embodiment of the present invention, since both the foreground color value and the background color value are HSV color values, the distance between the H components in the background color values ​​of the two regions to be merged and the distance between the H components in the foreground color values ​​of the two regions to be merged are calculated. If there is no second-class punctuation mark area in the two regions to be merged, the color values ​​should be slightly different, so the first color threshold (such as 50, 60, etc.) is selected. If the distance between the H components in the foreground color values ​​of the two regions to be merged and the distance between the H components in the background color values ​​are both less than the first color threshold, it is determined that the colors of the two regions to be merged are similar; otherwise, it is determined that the colors of the two regions to be merged are not similar. If there is at least one second-class punctuation mark area in the two regions to be merged, due to the small area of ​​the punctuation marks, combined with the character rendering mechanism, it will bring a large error, so the comparison threshold of the color value can be appropriately increased, and the second color threshold (such as 150, 160, etc.) is selected. If the distance between the H components in the foreground color values ​​of the two regions to be merged and the distance between the H components in the background color values ​​are both less than the second color threshold, it is determined that the colors of the two regions to be merged are similar; otherwise, it is determined that the colors of the two regions to be merged are not similar.

[0138] Step 319: Check whether the distance between the two regions to be merged is close. If yes, execute step 320; if no, execute step 321.

[0139] Step 320: determine that the two regions to be merged belong to the same text region, and merge the two regions to be merged.

[0140] When it is determined for the first time that two of the regions to be merged belong to the same text region, the two regions to be merged are merged, and the image attribute features and coordinate positions of the merged regions to be merged are obtained, and then the process returns to step 314 and traverses again.

[0141] Step 321: determine that the two regions to be merged do not belong to the same text region.

[0142] Step 322, whether the traversal is complete. If yes, then end; if no, then return to step 315.

[0143] After traversing all the regions to be merged, if any two regions to be merged do not belong to the same text region, it is determined that the merging is not continued, and at this time, the current region to be merged is regarded as a text region.

[0144] In the embodiment of the present invention, a series of principles for merging single character regions are proposed, such as the judgment of color, position, size, and punctuation marks. By performing binary classification on the color values, the foreground color and the background color are obtained, which provides a basis for color judgment. By introducing a text detection model, a unified third-party standard is provided for the judgment of punctuation marks. The text region obtained by merging single character regions is more accurate than the text region directly extracted, provides accurate information for subsequent code conversion, and ensures the availability of the code.

[0145] Figure 4 This is an effect diagram of extracting a text area in an embodiment of the present invention. Figure 7 FIG. 1 is another effect diagram of extracting text area in an embodiment of the present invention. Figure 4 and Figure 7 It can be seen that among the extracted text areas, some texts at the edge positions are not extracted into the text areas, some places that should not be extracted as text areas are extracted into text areas, and some places that should be extracted as text areas are not extracted into text areas.

[0146] Figure 5 This is an effect diagram of extracting a single character area in an embodiment of the present invention. Figure 8 FIG. 1 is another effect diagram of extracting a single character area in an embodiment of the present invention. Figure 5 and Figure 8 It can be seen that in the extracted single-character area, each single character is selected more accurately.

[0147] Figure 6 The figure is a diagram showing the effect of character merging in an embodiment of the present invention. Fig. 9 FIG. 1 is another effect diagram after character merging in an embodiment of the present invention. Figure 6 and Fig. 9 It can be seen that in the text area obtained by merging single-character areas, there is no text at the edge that is not in the text area, or the edge of the text area contains an area that is not text, and the merging effect is better.

[0148] Fig.10 The figure is a structural block diagram of a character merging device in an embodiment of the present invention.

[0149] like Fig.10 As shown, the character merging device may include the following modules:

[0150] Extraction module 1001, used for extracting a single character region from the image to be processed;

[0151] An acquisition module 1002 is used to acquire, for each single character region, image attribute features and coordinate positions of the single character region;

[0152] The merging module 1003 is used to merge the single character regions belonging to the same text region according to the image attribute features and coordinate positions of the single character regions.

[0153] Fig.11 It is a structural block diagram of another character merging device in an embodiment of the present invention.

[0154] like Fig.11 As shown, the character merging device may include the following modules:

[0155] An extraction module 111 is used to extract a single character area from the image to be processed;

[0156] An acquisition module 112, for acquiring image attribute features and coordinate positions of each single character region;

[0157] The merging module 113 is used to merge the single character regions belonging to the same text region according to the image attribute features and coordinate positions of the single character regions.

[0158] Optionally, the merging module 113 includes: an initialization submodule 1131, used to treat a single character area as an area to be merged; an area merging submodule 1132, used to repeatedly execute the following steps: for every two areas to be merged, judging whether the two areas to be merged belong to the same text area based on the image attribute characteristics and coordinate positions of the two areas to be merged; when it is judged for the first time that the two areas to be merged belong to the same text area, merging the two areas to be merged into one area to be merged, and obtaining the image attribute characteristics and coordinate positions of the merged area to be merged; an area determination submodule 1133, used to treat the current area to be merged as a text area when the merging is no longer to be continued.

[0159] Optionally, the image attribute features include color values; the region merging submodule 1132 includes: a first judgment unit 11321, used to judge whether the two regions to be merged overlap based on the coordinate positions of the two regions to be merged; when the first judgment unit judges that there is an overlap, judge that the two regions to be merged belong to the same text region; a second judgment unit 11322, used to judge whether the two regions to be merged are located in the same row or column based on the coordinate positions of the two regions to be merged when the first judgment unit judges that there is no overlap; a third judgment unit 11323, used to judge whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged when the second judgment unit judges that they are located in the same row or column, and judge whether the distance between the two regions to be merged is similar based on the coordinate positions of the two regions to be merged; when the colors and the distances are similar, judge that the two regions to be merged belong to the same text region.

[0160] Optionally, the device also includes: a first recognition module 114, used to identify whether the single-character area is a first-class punctuation area for each single-character area before the merging module merges the single-character areas belonging to the same text area according to the image attribute characteristics and coordinate positions of the single-character areas; the third judgment unit 11323 is specifically used to judge whether there is at least one first-class punctuation area in the two areas to be merged; if not, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged, and when the colors and the distances are similar, judging that the two areas to be merged belong to the same text area; if so, judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged, and when the distances are similar, judging that the two areas to be merged belong to the same text area.

[0161] Optionally, the image attribute features also include width and height; the first recognition module 114 includes: a first extraction submodule 1141, used to extract a control text area from the image to be processed, and obtain the coordinate position and height of the control text area; a first acquisition submodule 1142, used to select a control text area that overlaps with any single character area based on the coordinate position of any single character area and the coordinate position of the control text area, and obtain the maximum height of the selected control text area; a first comparison submodule 1143, used to determine that any single character area is a first-category punctuation area if both the width and height of any single character area are less than a first preset percentage of the maximum height.

[0162] Optionally, the device also includes: a second recognition module 115, which is used to identify, for each single-character area, whether the single-character area is a second-category punctuation area before the merging module merges the single-character areas belonging to the same text area according to the image attribute characteristics and coordinate positions of the single-character areas; the third judgment unit 11323 is specifically used to judge whether there is at least one second-category punctuation area in the two areas to be merged when judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged; if not, a preset first color threshold is selected to judge whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the first color threshold; if so, a preset second color threshold is selected to judge whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the second color threshold; the first color threshold is less than the second color threshold.

[0163] Optionally, the image attribute features also include width and height; the second recognition module 115 includes: a second extraction submodule 1151, used to extract a control text area from the image to be processed, and obtain the coordinate position and height of the control text area; a second acquisition submodule 1152, used to select a control text area that overlaps with any single character area based on the coordinate position of any single character area and the coordinate position of the control text area, and obtain the maximum height of the selected control text area; a second comparison submodule 1153, used to determine that any single character area is a second type of punctuation area if the width or height of any single character area is less than a second preset percentage of the maximum height.

[0164] Optionally, the color value includes a foreground color value and a background color value; the acquisition module 112 includes: a first color acquisition submodule 1121, used to obtain the RGB color value of the pixel for each pixel in any single-character area; a clustering submodule 1122, used to cluster all pixels according to the RGB color value of the pixel to obtain two clusters; a second color acquisition submodule 1123, used to obtain the RGB color value corresponding to the cluster for each cluster; a first color determination submodule 1124, used to convert the RGB color value corresponding to the cluster into an HSV color value, and use the HSV color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and use the HSV color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; the third judgment unit 11323 is specifically used to judge whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged.

[0165] Optionally, the color value includes a foreground color value and a background color value; the acquisition module 112 includes: a first color acquisition submodule 1121, used to obtain the RGB color value of the pixel for each pixel in any single-character area; a clustering submodule 1122, used to cluster all pixels according to the RGB color value of the pixel to obtain two clusters; a second color acquisition submodule 1123, used to obtain the RGB color value corresponding to each cluster for each cluster; a second color determination submodule 1125, used to use the RGB color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and use the RGB color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; the third judgment unit 11323 is specifically used to judge whether the colors of the two areas to be merged are similar based on the background color value and foreground color value of the two areas to be merged when judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged.

[0166] In the embodiment of the present invention, after the single character regions are merged based on the image attribute features and coordinate positions of the single character regions, the text region obtained has higher accuracy.

[0167] The embodiment of the present invention further provides an electronic device, such as Fig.12 As shown, it includes a processor 121 , a communication interface 122 , a memory 123 and a communication bus 124 , wherein the processor 121 , the communication interface 122 , and the memory 123 communicate with each other via the communication bus 124 .

[0168] Memory 123, used for storing computer programs;

[0169] The processor 121 is used to execute the program stored in the memory 123 to implement the following steps:

[0170] Extracting a single character region from the image to be processed;

[0171] For each single-character region, respectively obtain the image attribute features and coordinate position of the single-character region;

[0172] The single-character regions belonging to the same text region are merged according to the image attribute features and coordinate positions of the single-character regions.

[0173] Optionally, the single-character areas belonging to the same text area are merged based on the image attribute characteristics and coordinate positions of the single-character areas, including: taking a single-character area as a region to be merged; repeating the following steps: for every two regions to be merged, judging whether the two regions to be merged belong to the same text area based on the image attribute characteristics and coordinate positions of the two regions to be merged; when it is judged for the first time that the two regions to be merged belong to the same text area, merging the two regions to be merged into one region to be merged, and obtaining the image attribute characteristics and coordinate positions of the merged region to be merged; when the merging is no longer to be continued, taking the current region to be merged as a text area.

[0174] Optionally, the image attribute features include color values; and judging whether the two areas to be merged belong to the same text area based on the image attribute features and coordinate positions of the two areas to be merged includes: judging whether the two areas to be merged overlap based on the coordinate positions of the two areas to be merged; when there is overlap, judging that the two areas to be merged belong to the same text area; when there is no overlap, judging whether the two areas to be merged are located in the same row or column based on the coordinate positions of the two areas to be merged; when they are located in the same row or column, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distance between the two areas to be merged is similar based on the coordinate positions of the two areas to be merged; when the colors and the distance are similar, judging that the two areas to be merged belong to the same text area.

[0175] Optionally, before merging single-character areas belonging to the same text area based on the image attribute features and coordinate positions of the single-character areas, it also includes: for each single-character area, identifying whether the single-character area is a first-category punctuation area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged; when the colors and the distances are similar, judging that the two areas to be merged belong to the same text area, including: judging whether there is at least one first-category punctuation area in the two areas to be merged; if not, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged; when the colors and the distances are similar, judging that the two areas to be merged belong to the same text area; if so, judging whether the distances of the two areas to be merged are similar based on the coordinate positions of the two areas to be merged; when the distances are similar, judging that the two areas to be merged belong to the same text area.

[0176] Optionally, the image attribute features also include width and height; for any single-character area, identifying whether the any single-character area is a first-category punctuation area, including: extracting a control text area from the image to be processed, and obtaining the coordinate position and height of the control text area; based on the coordinate position of the any single-character area and the coordinate position of the control text area, selecting a control text area that overlaps with the any single-character area, and obtaining the maximum height of the selected control text area; if the width and height of the any single-character area are both less than a first preset percentage of the maximum height, determining that the any single-character area is a first-category punctuation area.

[0177] Optionally, before merging single-character areas belonging to the same text area based on the image attribute features and coordinate positions of the single-character areas, it also includes: for each single-character area, separately identifying whether the single-character area is a second-category punctuation area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, including: judging whether there is at least one second-category punctuation area in the two areas to be merged; if not, selecting a preset first color threshold, and judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the first color threshold; if so, selecting a preset second color threshold, and judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged and the second color threshold; the first color threshold is less than the second color threshold.

[0178] Optionally, the image attribute features also include width and height; for any single-character area, identifying whether the any single-character area is a second-category punctuation area, including: extracting a control text area from the image to be processed, and obtaining the coordinate position and height of the control text area; based on the coordinate position of the any single-character area and the coordinate position of the control text area, selecting a control text area that overlaps with the any single-character area, and obtaining the maximum height of the selected control text area; if the width or height of the any single-character area is less than a second preset percentage of the maximum height, determining that the any single-character area is a second-category punctuation area.

[0179] Optionally, the color value includes a foreground color value and a background color value; for any single-character area, obtaining the color value of any single-character area, including: for each pixel in the any single-character area, obtaining the RGB color value of the pixel; clustering all pixels according to the RGB color value of the pixel to obtain two clusters; for each cluster, respectively obtaining the RGB color value corresponding to the cluster; converting the RGB color value corresponding to the cluster into an HSV color value, using the HSV color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and using the HSV color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, including: judging whether the colors of the two areas to be merged are similar based on the hue H component in the background color value and the hue H component in the foreground color value of the two areas to be merged.

[0180] Optionally, the color value includes a foreground color value and a background color value; for any single-character area, obtaining the color value of any single-character area, including: for each pixel in the any single-character area, obtaining the RGB color value of the pixel; clustering all pixels according to the RGB color value of the pixel to obtain two clusters; for each cluster, respectively obtaining the RGB color value corresponding to the cluster; using the RGB color value corresponding to the cluster with a large number of pixels as the background color value of any single-character area, and using the RGB color value corresponding to the cluster with a small number of pixels as the foreground color value of any single-character area; judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, including: judging whether the colors of the two areas to be merged are similar based on the background color value and foreground color value of the two areas to be merged.

[0181] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0182] The communication interface is used for communication between the above terminal and other devices.

[0183] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0184] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0185] In another embodiment provided by the present invention, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer implements the character merging method described in any one of the above embodiments.

[0186] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to implement any of the character merging methods described in the above embodiments.

[0187] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0188] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0189] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0190] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A character merging method, characterized in that: include: Extracting a single character region from the image to be processed; For each single-character region, respectively obtain the image attribute features and coordinate position of the single-character region; The image attribute features include color values; Merging the single character regions belonging to the same text region according to the image attribute features and coordinate positions of the single character regions includes: Treat a single character region as a region to be merged; Repeat the following steps: for each two regions to be merged, based on the coordinate positions of the two regions to be merged, determine whether the two regions to be merged overlap; When there is overlap, determining that the two regions to be merged belong to the same text region; When there is no overlap, judging whether the two regions to be merged are located in the same row or the same column based on the coordinate positions of the two regions to be merged; When the two regions are located in the same row or the same column, judging whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged, and judging whether the distances of the two regions to be merged are similar based on the coordinate positions of the two regions to be merged; when the colors and the distances are similar, judging that the two regions to be merged belong to the same text area, including: Determine whether there is at least one first-category punctuation mark region in the two regions to be merged; If not, judging whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged, and judging whether the distances of the two regions to be merged are similar based on the coordinate positions of the two regions to be merged; when the colors and the distances are similar, judging that the two regions to be merged belong to the same text region; If so, based on the coordinate positions of the two regions to be merged, it is determined whether the distance between the two regions to be merged is close; when the distance is close, it is determined that the two regions to be merged belong to the same text region.

2. The method according to claim 1, characterized in that When it is determined for the first time that the two regions to be merged belong to the same text region, the two regions to be merged are merged into one region to be merged, and image attribute features and coordinate positions of the merged region to be merged are obtained; When the merging is no longer continued, the current area to be merged is used as a text area.

3. The method according to claim 1, characterized in that Before merging the single-character regions belonging to the same text region according to the image attribute features and coordinate positions of the single-character regions, the method further includes: for each single-character region, identifying whether the single-character region is a first-category punctuation mark region.

4. The method according to claim 3, characterized in that The image attribute feature also includes width and height; for any single character area, identifying whether the any single character area is a first type of punctuation mark area includes: Extracting a reference text area from the image to be processed, and obtaining the coordinate position and height of the reference text area; Based on the coordinate position of any one of the single-character regions and the coordinate position of the comparison text region, selecting a comparison text region that overlaps with any one of the single-character regions, and obtaining the maximum height of the selected comparison text region; If the width and height of any one of the single-character regions are both smaller than a first preset percentage of the maximum height, the any one of the single-character regions is determined to be a first-category punctuation mark region.

5. The method according to claim 1, characterized in that: Before merging the single-character regions belonging to the same text region according to the image attribute features and coordinate positions of the single-character regions, the method further includes: for each single-character region, identifying whether the single-character region is a second-category punctuation mark region; The determining, based on the color values ​​of the two regions to be merged, whether the colors of the two regions to be merged are similar includes: Determine whether there is at least one second-category punctuation mark region in the two regions to be merged; If not, selecting a preset first color threshold, and judging whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged and the first color threshold; If yes, then selecting a preset second color threshold, and judging whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged and the second color threshold; The first color threshold is less than the second color threshold.

6. The method according to claim 5, characterized in that The image attribute feature also includes width and height; for any single character area, identifying whether the any single character area is a second type of punctuation mark area includes: Extracting a reference text area from the image to be processed, and obtaining the coordinate position and height of the reference text area; Based on the coordinate position of any one of the single-character regions and the coordinate position of the comparison text region, selecting a comparison text region that overlaps with any one of the single-character regions, and obtaining the maximum height of the selected comparison text region; If the width or height of any one of the single-character regions is smaller than a second preset percentage of the maximum height, the any one of the single-character regions is determined to be a second-category punctuation mark region.

7. The method according to claim 1, characterized in that The color value includes a foreground color value and a background color value; For any single-character region, obtaining the color value of the any single-character region includes: For each pixel in any single-character area, obtaining the RGB color value of the pixel; Clustering all pixels according to the RGB color values ​​of the pixels to obtain two clusters; For each cluster, respectively obtain the RGB color value corresponding to the cluster; Convert the RGB color values ​​corresponding to the clusters into HSV color values, use the HSV color values ​​corresponding to the clusters with more pixels as the background color values ​​of any single-character region, and use the HSV color values ​​corresponding to the clusters with fewer pixels as the foreground color values ​​of any single-character region; The determining whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged includes: determining whether the colors of the two areas to be merged are similar based on the hue H component in the background color values ​​and the hue H component in the foreground color values ​​of the two areas to be merged.

8. The method according to claim 1, characterized in that The color value includes a foreground color value and a background color value; For any single-character region, obtaining the color value of the any single-character region includes: For each pixel in any single-character area, obtaining the RGB color value of the pixel; Clustering all pixels according to the RGB color values ​​of the pixels to obtain two clusters; For each cluster, respectively obtain the RGB color value corresponding to the cluster; The RGB color value corresponding to the cluster with more pixels is used as the background color value of any single character area, and the RGB color value corresponding to the cluster with less pixels is used as the foreground color value of any single character area; The determining whether the colors of the two regions to be merged are similar based on the color values ​​of the two regions to be merged includes: determining whether the colors of the two regions to be merged are similar based on the background color values ​​and the foreground color values ​​of the two regions to be merged.

9. A character merging device, characterized in that: include: An extraction module, used for extracting a single character region from the image to be processed; An acquisition module, for acquiring, for each single-character region, image attribute features and coordinate positions of the single-character region; the image attribute features include color values; A merging module is used to merge single-character areas belonging to the same text area according to the image attribute features and coordinate positions of the single-character areas, including: taking a single-character area as an area to be merged; repeatedly performing the following steps: for each two areas to be merged, judging whether the two areas to be merged overlap based on the coordinate positions of the two areas to be merged; when there is overlap, judging that the two areas to be merged belong to the same text area; when there is no overlap, judging whether the two areas to be merged are located in the same row or column based on the coordinate positions of the two areas to be merged; when they are located in the same row or column, judging whether the colors of the two areas to be merged are similar based on the color values ​​of the two areas to be merged, and judging based on the coordinate positions of the two areas to be merged whether the distance between the two areas to be merged is close; when the color and the distance are both close, judging that the two areas to be merged belong to the same text area, including: judging whether there is at least one first-category punctuation mark area in the two areas to be merged; if not, judging whether the colors of the two areas to be merged are close based on the color values ​​of the two areas to be merged, and judging whether the distance between the two areas to be merged is close based on the coordinate positions of the two areas to be merged; when the color and the distance are both close, judging that the two areas to be merged belong to the same text area; if so, judging whether the distance between the two areas to be merged is close based on the coordinate positions of the two areas to be merged; when the distance is close, judging that the two areas to be merged belong to the same text area.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 8 when executing a program stored in a memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images