Image text area positioning method and device, storage medium and electronic device

Through pre-processing and positioning strategies for different image types, including expansion processing, linear detection and single-word recognition models, the accuracy and universality of image text area positioning in the prior art are solved, and the accuracy of various text images is achieved. position.

CN112560847BActive Publication Date: 2024-12-06CHINA CONSTRUCTION BANK
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202011561668.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-25
Publication Date
2024-12-06
Estimated Expiration
2040-12-25

AI Technical Summary

Technical Problem

Existing image text area positioning methods are difficult to accurately locate left and right edge positions in plain text images, especially when there is fewer text lines or noise, and cannot effectively process text images of complex background layout types.

Method used

Different preprocessing and positioning strategies are carried out according to the image type, including expanding the plain text image to identify the text line connection area, horizontal and vertical lines are detected for text line interleaving types, and single-word recognition model is used for complex background layout types to merge single-word and perform linear frame detection.

Benefits of technology

Accurate positioning of different types of text images, including the left and right edge positions of plain text images, enhance the accuracy and versatility of image text area positioning, and can handle text images in complex backgrounds.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112560847B_ABST
    Figure CN112560847B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for locating an image text area, a storage medium and an electronic device. For a text image of a pure text type, the text image is expanded so that adjacent characters are connected into a text line connected area, and the image text area is determined by the circumscribed rectangle of the text line connected area; for a text image of a text straight line staggered type, the image text area is determined by detecting a straight line frame in the text image; for a text image of a complex background layout type, the single word frame in the text image is identified, and the single word frame is merged into a text line connected area, and the image text area is determined by the text line connected area and the detected straight line frame. It can be seen that by identifying the straight line frame and / or the circumscribed rectangle of the connected area in the text image, the upper, lower, left and right edge positions of the text line in the text image can be accurately located, and the image text area positioning is universal for various types of text images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for locating an image text area, a storage medium, and an electronic device. Background Art

[0002] Image text recognition is widely needed in many fields, and related applications include ID card recognition, license plate number recognition, express delivery order recognition, bank card number recognition, etc. Image text recognition refers to the recognition of text in text images. Text images are usually divided into pure text type images, text straight line interlaced type images (such as table images) and complex background layout type images (such as bill images), and image text area positioning is the prerequisite for image text recognition.

[0003] At present, the solution for locating image text areas is mainly the image pixel projection analysis method, which specifically includes: binarizing the image to make the text in the image black and the background white, horizontally projecting the pixels in the image, calculating the number of black pixels on each text line, and thus obtaining a pixel distribution map, and by setting a threshold, determining the starting and ending points of each peak on the pixel distribution map as the upper and lower boundaries of the text line according to the threshold.

[0004] The existing image text area localization scheme can only locate the upper and lower edge positions of each line of text in a pure text image, but it is difficult to locate the left and right edge positions. In addition, for text lines with only a small number of characters, the peak value after horizontal projection will be relatively small, which will cause difficulties in subsequent threshold setting. If the threshold is set too small, noise will be identified as text lines, and if the threshold is set too large, text lines with a small number of single words will be ignored. In addition, this scheme cannot locate image text areas for text images with straight lines. Summary of the invention

[0005] The present application provides a method and device for locating an image text region, a storage medium and an electronic device, the purpose of which is to improve the accuracy and versatility of image text region positioning.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A method for locating an image text region, comprising:

[0008] Acquire a text image to be located, and determine the image type of the text image to be located; the image type includes a pure text type, a text straight line staggered type, or a complex background layout type;

[0009] If the image type of the to-be-located text image is a pure text type, the to-be-located text image is subjected to image preprocessing, and the to-be-located text image after image preprocessing is subjected to dilation processing to obtain a target text image, and each text line connected area in the target text image is identified, and the coordinate value of the circumscribed rectangle of each of the text line connected areas is determined, and based on the coordinate value of the circumscribed rectangle of each of the text line connected areas, the text line area in the to-be-located text image is determined; wherein the pixel values ​​of adjacent pixel points in each of the text line connected areas are the same;

[0010] If the image type of the to-be-located text image is a text straight line interlaced type, the to-be-located text image is subjected to image preprocessing, and horizontal line detection and vertical line detection are performed on the to-be-located text image after image preprocessing, and multiple rectangles are determined based on each horizontal line and each vertical line obtained by detection, and a text line area in the to-be-located text image is determined according to the coordinate values ​​of the rectangles;

[0011] If the image type of the text image to be located is a complex background layout type, the text image to be located is input into a pre-constructed single-word recognition model to obtain the coordinate prediction value and confidence of the single-word frame corresponding to each single word in the text image to be located, and based on the confidence of each single-word frame, multiple target single-word frames are determined from each of the single-word frames, and the target single-word frames adjacent in the horizontal direction are merged to obtain multiple text line connected areas, and horizontal line detection and vertical line detection are performed on the text image to be located. According to each of the text line connected areas and the detected horizontal lines and vertical lines, the text line area in the text image to be located is determined.

[0012] In the above method, optionally, the image preprocessing of the to-be-located text image includes:

[0013] Performing grayscale processing on the image of the text to be located to obtain a grayscale image;

[0014] Performing filtering on the grayscale image to obtain a filtered image;

[0015] Performing adaptive binarization processing on the filtered image to obtain a binarized image;

[0016] The pixel value of each pixel in the binary image is inverted.

[0017] In the above method, optionally, filtering the grayscale image to obtain a filtered image includes:

[0018] Slide the center of a preset filtering sliding window across each pixel in the grayscale image;

[0019] Each time the center of the filtering sliding window slides to a pixel point in the grayscale image, based on the noise type of the text image to be located, a preset filtering calculation formula corresponding to the noise type is selected, and based on the selected filtering calculation method, the filtering grayscale value within the current filtering sliding window is calculated, and the calculated filtering grayscale value is used as the pixel value of the pixel point.

[0020] In the above method, optionally, the step of performing expansion processing on the image to be located after image preprocessing to obtain the target text image includes:

[0021] Based on the first sliding window, the text image to be located after image preprocessing is expanded; wherein the width of the first sliding window is determined according to the spacing between adjacent characters in the text image to be located, and the height of the first sliding window is determined according to the line spacing of text lines in the image to be located.

[0022] In the above method, optionally, the step of performing horizontal line detection on the image of the text to be located after image preprocessing comprises:

[0023] Based on a preset second sliding window, the image of the text to be located is corroded after image preprocessing to obtain a first corroded image;

[0024] Based on a preset third sliding window, dilating the first eroded image to obtain a first dilated image;

[0025] Identify each horizontally connected region in the first dilated image, and determine a circumscribed rectangle of each horizontally connected region;

[0026] For each of the horizontally connected regions, the coordinates of two end points of a horizontal line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the horizontally connected region.

[0027] In the above method, optionally, the step of performing vertical line detection on the image of the text to be located after image preprocessing includes:

[0028] Based on a preset fourth sliding window, the image of the text to be located is corroded after image preprocessing to obtain a second corroded image;

[0029] Based on a preset fifth sliding window, dilating the second eroded image to obtain a second dilated image;

[0030] identifying each vertically connected region in the second dilated image, and determining a circumscribed rectangle of each vertically connected region;

[0031] For each of the vertically connected regions, the coordinates of two end points of a vertical line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the vertically connected region.

[0032] In the above method, optionally, the step of determining a plurality of target single-word frames from each of the single-word frames based on the confidence of each of the single-word frames comprises:

[0033] For each of the single-word frames, if the confidence of the single-word frame is not less than a preset confidence threshold, the single-word frame is determined as an initial single-word frame;

[0034] Grouping the initial single-word frames into a single-word frame set;

[0035] Selecting a first single-word frame from the current single-word frame set; wherein the first single-word frame is an initial single-word frame with the largest confidence among the initial single-word frames included in the current single-word frame set;

[0036] For each remaining initial single-character frame in the single-character frame set, calculating an area overlap ratio between the initial single-character frame and the first single-character frame, and if the area overlap ratio is greater than a preset overlap threshold, deleting the initial single-character frame from the single-character frame set;

[0037] Determine the target single-word frame by using the first single-word frame, and determine whether the current single-word frame set is an empty set;

[0038] If the current single-character frame set is not an empty set, the process returns to the step of selecting the first single-character frame from the current single-character frame set until the current single-character frame set is an empty set.

[0039] An image text area positioning device, comprising:

[0040] An acquisition unit, used for acquiring a text image to be located, and determining an image type of the text image to be located; the image type includes a pure text type, a text straight line interlaced type, or a complex background layout type;

[0041] A first positioning unit is used for, if the image type of the text image to be positioned is a pure text type, performing image preprocessing on the text image to be positioned, and performing dilation processing on the text image to be positioned after the image preprocessing to obtain a target text image, identifying each text line connected area in the target text image, determining the coordinate value of the circumscribed rectangle of each text line connected area, and determining the text line area in the text image to be positioned based on the coordinate value of the circumscribed rectangle of each text line connected area; wherein the pixel values ​​of adjacent pixel points in each text line connected area are the same;

[0042] A second positioning unit is used for, if the image type of the text image to be positioned is a text straight line interlaced type, performing image preprocessing on the text image to be positioned, and performing horizontal line detection and vertical line detection on the text image to be positioned after the image preprocessing, determining a plurality of rectangles based on each horizontal line and each vertical line obtained by the detection, and determining a text line area in the text image to be positioned according to the coordinate values ​​of the rectangles;

[0043] The third positioning unit is used for inputting the text image to be positioned into a pre-constructed single-word recognition model if the image type of the text image to be positioned is a complex background layout type, obtaining the coordinate prediction value and confidence of the single-word frame corresponding to each single word in the text image to be positioned, determining a plurality of target single-word frames from each of the single-word frames based on the confidence of each of the single-word frames, merging the target single-word frames adjacent in the horizontal direction to obtain a plurality of text line connected areas, performing horizontal line detection and vertical line detection on the text image to be positioned, and determining the text line area in the text image to be positioned according to each of the text line connected areas and the detected horizontal lines and vertical lines.

[0044] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned image text area positioning method.

[0045] An electronic device comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the above-mentioned image text area positioning method.

[0046] Compared with the prior art, this application has the following advantages:

[0047] The present application provides a method and device for locating an image text area, the method comprising: for different types of text images, adopting different image text area positioning strategies, for a text image of a pure text type, by dilating the text image, so that adjacent characters are connected into a text line connected area, and then determining the circumscribed rectangle of the text line connected area, so as to realize the positioning of the area where the text exists in the text image; for a text image of a text straight line staggered type, by detecting the straight line frame in the text image, the area where the text exists in the text image is positioned, for a text image of a complex background layout type, based on a single word recognition model, the single word frame corresponding to each single word in the text image is identified, the single word frame is merged into a text line connected area, and by detecting the straight line frame in the text image, the area where the text exists in the text image is positioned through the straight line frame and the text line connected area. It can be seen that the technical solution provided by the present application realizes the accurate positioning of the upper, lower, left and right edge positions of each text line in the text image by identifying the straight line frame and / or the circumscribed rectangle of the connected area in the text image, and adopts different image text area positioning strategies for different types of text images, so as to realize the image text area positioning of each type of text image with universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0049] Figure 1 A method flow chart of a method for locating text area in an image provided by this application;

[0050] Figure 2 A flowchart of another method for locating an image text area provided by the present application;

[0051] Figure 3 A flowchart of another method for locating an image text area provided by the present application;

[0052] Figure 4 A flowchart of another method for locating an image text area provided by the present application;

[0053] Figure 5 A flowchart of another method for locating an image text area provided by the present application;

[0054] Figure 6 A schematic diagram of the structure of an image text area positioning device provided in this application;

[0055] Figure 7 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.

[0058] The embodiment of the present application provides a method for locating an image text area. The method can be applied to a variety of system platforms, and its execution subject can run on a computer terminal or a processor of various mobile devices. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0059] S101, obtaining a text image to be located, and determining the image type of the text image to be located.

[0060] The text image to be located is obtained, and the image type of the text image to be located is determined, wherein the image type includes a pure text image type, a text straight line interlaced type, or a complex background layout type.

[0061] Optionally, the specific process of determining the image type of the text image to be located includes: receiving the image type of the text image to be located uploaded by the user, or inputting the text image to be located into a pre-built image recognition model to obtain the image type of the text image to be located output by the image recognition model. Optionally, the image recognition model can be a classification model, and the specific construction process refers to the construction process of the existing convolutional neural network classification model.

[0062] S102: If the image type of the text image to be located is a plain text type, perform image preprocessing on the text image to be located.

[0063] If the image type of the text image to be located is a pure text type, the text image to be located is subjected to image preprocessing. Optionally, the image preprocessing includes grayscale processing, filtering processing, adaptive binarization processing and pixel inversion processing on the text image to be processed to enhance the image quality of the text image to be processed.

[0064] See also Figure 2 , the process of image preprocessing for the text image to be located specifically includes:

[0065] S201 , grayscale the text image to be located to obtain a grayscale image.

[0066] The text image to be located is grayed to obtain a grayed image of the text image to be located. The specific grayed processing process includes: according to a preset grayed formula, each pixel in the text image to be located is grayed to obtain a grayed image of the text image to be located. In the grayed image, each pixel is represented by a value between 0 and 255 to represent the depth of gray.

[0067] Optionally, the preset grayscale formula is as follows:

[0068] GYAY=R×0.299+G×0.587+B×0.114

[0069] Among them, R, G and B represent the values ​​of red, green and blue respectively, and GRAY is the final grayscale value.

[0070] S202: Filter the grayscale image to obtain a filtered image.

[0071] The grayscaled image of the text to be located is subjected to filtering, that is, the grayscaled image is subjected to filtering to obtain a filtered image of the text to be located. The specific filtering process may include:

[0072] Slide the center of the preset filter sliding window across each pixel in the grayscale image;

[0073] Every time the center of the filtering sliding window slides to a pixel point in the grayscale image, based on the noise type of the text image to be located, a preset filtering calculation formula corresponding to the noise type is selected, and based on the selected filtering calculation method, the filtering grayscale value in the current filtering sliding window is calculated, and the calculated filtering grayscale value is used as the pixel value of the pixel point.

[0074] In the method provided in the embodiment of the present application, the center of the preset filter sliding window passes through each pixel in the grayscale image, that is, based on the preset sliding mode, the preset filter sliding window slides in the grayscale image so that the center of the filter sliding window passes through each pixel in the grayscale image. The preset sliding mode is a manually set sliding mode and is not limited here.

[0075] In the method provided in the embodiment of the present application, each time a pixel is slid to a point, a preset filtering calculation method corresponding to the noise type is selected based on the noise type of the text image to be located, and the grayscale value in the current filtering sliding window is calculated based on the selected filtering calculation method, and the calculated filtering grayscale value is used as the pixel value of the pixel point. If the noise type of the text image to be located is a white noise type, the grayscale value in each filtering sliding window is calculated using the Gaussian filtering calculation formula, and if the noise type of the text image to be located is a salt and pepper noise type, the grayscale value in each filtering sliding window is calculated using the median filtering calculation formula.

[0076] S203 , performing adaptive binarization processing on the filtered image to obtain a binarized image.

[0077] Each pixel in the filtered image is subjected to adaptive binarization processing, and the grayscale value of each pixel in the filtered image is 0 or 255, thereby obtaining a binarized image of the text image to be located, where 0 represents black and 255 represents white.

[0078] Optionally, the specific process of performing adaptive binarization processing on the filtered image includes:

[0079] Each pixel point in the filtered image is marked at the center of a preset binary sliding window, and each time a pixel point is slid to, the current binarization threshold is calculated based on the pixel values ​​of all the pixels in the current binary sliding window, and the pixel value of the central pixel point in the current binary sliding window is compared with the current binarization threshold. If the pixel value of the central pixel point is greater than the binarization threshold, a preset first value is used as the pixel value of the central pixel point in the current binary sliding window; if the pixel value of the central pixel point is not greater than the binarization threshold, a preset second value is used as the pixel value of the central pixel point in the current binary sliding window; wherein the first value is 255 and the second value is 0.

[0080] In the method provided in the embodiment of the present application, the filtered image is adaptively binarized so that the entire image presents only two pixel grayscale values, black and white, thereby highlighting the target outline, that is, highlighting the outline of each text line. The adaptive binarization processing method provided in the present application performs adaptive binarization on the filtered image so that the binarization threshold corresponding to each pixel point in the filtered image is not fixed, but is determined by the pixel values ​​of all pixels in the binary sliding window. The binarization threshold of the image area with higher brightness is usually higher, while the binarization threshold of the image area with lower brightness will be correspondingly lower, which can adapt to images with different brightness, different contrast, and different textures.

[0081] S204: Invert the pixel value of each pixel in the binary image.

[0082] The pixel value of each pixel in the binary image is inverted. If the pixel value of the pixel is the first value, it is inverted to the second value. If the pixel value of the pixel is the second value, it is inverted to the first value. That is, if the pixel value of the pixel is 255, it is inverted to 0. If the pixel value of the pixel is 0, it is inverted to 255, thereby achieving the color inversion of the black and white pixels in the binary image.

[0083] S103 , dilating the to-be-located text image after image preprocessing to obtain a target text image.

[0084] The image of the text to be located after image preprocessing is expanded to connect adjacent words in each line to obtain the target text image. The specific expansion process is as follows:

[0085] Based on the first sliding window, the text image to be located after image preprocessing is expanded; wherein the width of the first sliding window is determined according to the spacing between adjacent characters in the text image to be located, and the height of the first sliding window is determined according to the line spacing of text lines in the image to be located.

[0086] In the method provided in the embodiment of the present application, based on the first sliding window, the center of the first sliding window is slid over each pixel point of the text image to be located after image preprocessing in a preset sliding manner, and each time the center of the first sliding window slides to a pixel point of the text image to be located after image preprocessing, the maximum pixel value within the current coverage of the first sliding window is used as the pixel value of the pixel point corresponding to the center of the first sliding window to achieve expansion processing.

[0087] In the method provided in the embodiment of the present application, the width of the first sliding window is determined according to the spacing between adjacent characters in the text image to be located, and the height of the first sliding window is determined according to the line spacing between text lines in the text image to be located.

[0088] S104: Identify each text line connected region in the target text image, and determine the coordinate value of the circumscribed rectangle of each text line connected region.

[0089] Identify each text line connected region in the target text image, wherein the pixel values ​​of adjacent pixels in each text line connected region are the same.

[0090] Based on the identified text line connected areas, the coordinate values ​​of the circumscribed rectangle of each text line connected area are determined. That is, for each text line connected area, the circumscribed contour of the text line connected area is determined as the circumscribed rectangle of the text line connected area. After the circumscribed rectangle is determined, the coordinate values ​​of the circumscribed rectangle can be determined. If necessary, the coordinate values ​​of the circumscribed rectangle are the coordinate values ​​of the four corners of the circumscribed rectangle.

[0091] Optionally, after determining the circumscribed rectangle of each text connected area, the circumscribed rectangles with obviously excessive or small width may be further deleted, that is, the circumscribed rectangles with a height not within a preset height range and / or a width not within a preset width range may be deleted, while the circumscribed rectangles with a height within a preset height range and a width within a preset width range may be retained.

[0092] S105 . Determine the text line region in the text image to be located based on the coordinate values ​​of the circumscribed rectangle of each text line connected region.

[0093] Based on the coordinate values ​​of the circumscribed rectangles of the connected areas of each text line, the text line area in the text image to be located is determined. That is to say, the coordinate values ​​of each circumscribed rectangle can determine a rectangle, which corresponds to a text line area in the text image to be located, and the coordinate values ​​of all the circumscribed rectangles can determine all the text line areas in the text image to be located.

[0094] S106: If the image type of the text image to be located is a text straight line interlaced type, perform image preprocessing on the text image to be located.

[0095] If the image type of the text image to be located is a text straight line interlaced type, then similar to the text image of the pure text type, the text image to be located needs to be preprocessed. For the specific process of image preprocessing, please refer to the embodiment Figure 2 , I will not go into details here.

[0096] S107 , performing horizontal line detection and vertical line detection on the image of the text to be located after image preprocessing, and determining a plurality of rectangles based on each horizontal line and each vertical line obtained through the detection.

[0097] In the method provided in the embodiment of the present application, horizontal line detection and vertical line detection are performed on the image of the text to be located after image preprocessing, that is, the straight line frame in the image of the text to be located after image preprocessing is detected.

[0098] In the method provided in the embodiment of the present application, the horizontal lines in the text image to be located after image preprocessing are detected by performing horizontal erosion processing on the text image to be located after image preprocessing and performing horizontal expansion processing after the horizontal erosion processing.

[0099] See also Figure 3 , the process of performing horizontal line detection on the image to be located text after image preprocessing specifically includes:

[0100] S301 . Based on a preset second sliding window, an erosion process is performed on the image of the text to be located after image preprocessing to obtain a first eroded image.

[0101] In the method provided in the embodiment of the present application, the image of the text to be located after image preprocessing is corroded based on a preset second sliding window, that is, the image of the text to be located after image preprocessing is horizontally corroded to obtain a first corroded image. It should be noted that the specific process of the corrosion process can be referred to the existing image corrosion process, which will not be repeated here.

[0102] It should be noted that the aspect ratio of the second sliding window is greater than the first threshold. Optionally, the first threshold may be 30, that is, the aspect ratio of the second sliding window is greater than 30. Optionally, the height of the second sliding window satisfies a preset first height range, and the first height range is 1-2 pixels.

[0103] In the method provided in the embodiment of the present application, the second sliding window is used to perform corrosion processing on the image of the text to be located after image preprocessing, which can suppress vertical lines and other non-horizontal line image elements.

[0104] S302: Based on a preset third sliding window, dilate the first eroded image to obtain a first dilated image.

[0105] In the method provided in the embodiment of the present application, the first eroded image is expanded based on a preset third sliding window, that is, the first eroded image is horizontally expanded to obtain a first expanded image. It should be noted that the specific process of the expansion process can be referred to the existing image expansion process, which will not be repeated here.

[0106] It should be noted that the aspect ratio of the third sliding window is greater than the second threshold. Optionally, the second threshold may be 20, that is, the aspect ratio of the third sliding window is greater than 20. Optionally, the height of the second sliding window satisfies a preset second height range, and the second height range is 1-5 pixels.

[0107] S303 : Identify each horizontally connected region in the first dilated image, and determine a circumscribed rectangle of each horizontally connected region.

[0108] Identify each horizontally connected region in the first dilated image, where the horizontally connected region is the horizontally dilated region, determine the circumscribed contour of the horizontally connected region, and then determine the circumscribed rectangle of each horizontally connected region.

[0109] S304 . For each horizontally connected region, calculate the coordinates of two end points of a horizontal line corresponding to the circumscribed rectangle according to the coordinates of the circumscribed rectangle of the horizontally connected region.

[0110] For each horizontally connected region, the coordinates of the two end points of the horizontal line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the horizontally connected region.

[0111] The specific process of calculating the coordinates of the two endpoints of the horizontal line corresponding to the circumscribed rectangle of each horizontally connected region according to the coordinates of the circumscribed rectangle of the horizontally connected region includes:

[0112] Calculate the average of the ordinates of the upper and lower edges of the circumscribed rectangle of the horizontally connected region, and use the average as the ordinates of the two endpoints of the horizontal line corresponding to the circumscribed rectangle; use the abscissa of the left edge of the circumscribed rectangle as the abscissa of the left endpoint of the horizontal line corresponding to the circumscribed rectangle, and use the abscissa of the right edge of the circumscribed rectangle as the abscissa of the right endpoint of the horizontal line corresponding to the circumscribed rectangle. For example, the ordinate of the upper edge of the circumscribed rectangle is y up , the lower edge has a vertical coordinate of y down , the horizontal coordinate of the left edge is x left , the horizontal coordinate of the right edge is x right , then the left endpoint of the horizontal line corresponding to the circumscribed rectangle is The right endpoint is

[0113] In the method provided in the embodiment of the present application, the vertical lines in the text image to be located after image preprocessing are detected by performing vertical erosion processing on the text image to be located after image preprocessing and performing vertical expansion processing after the vertical erosion processing.

[0114] See also Figure 4 , the process of performing vertical line detection on the image of the text to be located after image preprocessing specifically includes:

[0115] S401 . Based on a preset fourth sliding window, an erosion process is performed on the image of the text to be located after image preprocessing to obtain a second eroded image.

[0116] In the method provided in the embodiment of the present application, the text image to be located after image preprocessing is corroded based on the preset fourth sliding window, that is, the text image to be located after image preprocessing is vertically corroded to obtain a second corroded image. It should be noted that the specific process of the corrosion process can be referred to the existing image corrosion process, which will not be repeated here.

[0117] It should be noted that the aspect ratio of the fourth sliding window is smaller than the third threshold. Optionally, the third threshold may be 1 / 30, that is, the aspect ratio of the fourth sliding window is smaller than 1 / 30. Optionally, the width of the fourth sliding window may be 1 pixel.

[0118] In the method provided in the embodiment of the present application, the fourth sliding window is used to perform corrosion processing on the image of the text to be located after image preprocessing, which can suppress horizontal lines and other non-vertical line image elements.

[0119] S402: Based on a preset fifth sliding window, dilate the second eroded image to obtain a second dilated image.

[0120] In the method provided in the embodiment of the present application, the second eroded image is expanded based on the preset fifth sliding window, that is, the first eroded image is vertically expanded to obtain a second expanded image. It should be noted that the specific process of the expansion process can be referred to the existing image expansion process, which will not be repeated here.

[0121] It should be noted that the aspect ratio of the fifth sliding window is smaller than the fourth threshold. Optionally, the fourth threshold may be 1 / 20, that is, the aspect ratio of the fifth sliding window is greater than 1 / 20. Optionally, the width of the fifth sliding window may be smaller than the fifth threshold, and the fifth threshold may be 5 pixels, that is, the width of the fifth sliding window may be smaller than 5 pixels.

[0122] S403: Identify each vertically connected region in the second dilated image, and determine a circumscribed rectangle of each vertically connected region.

[0123] Each vertically connected region in the second dilated image is identified, the vertically connected region is a vertically dilated region, and the circumscribed contour of the vertically connected region is determined, and then the circumscribed rectangle of each vertically connected region is determined.

[0124] S404: For each of the vertically connected regions, calculate the coordinates of two end points of a vertical line corresponding to the circumscribed rectangle according to the coordinates of the circumscribed rectangle of the vertically connected region.

[0125] For each vertically connected region, the coordinates of the two end points of the vertical line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the vertically connected region.

[0126] The specific process of calculating the coordinates of the two end points of the vertical line corresponding to the circumscribed rectangle of each vertically connected region according to the coordinates of the circumscribed rectangle of the vertically connected region includes:

[0127] Calculate the average of the horizontal coordinates of the left and right edges of the circumscribed rectangle of the vertically connected region, and use the average as the horizontal coordinates of the two endpoints of the vertical line corresponding to the circumscribed rectangle; use the vertical coordinate of the upper edge of the circumscribed rectangle as the vertical coordinate of the upper endpoint of the vertical line corresponding to the circumscribed rectangle, and use the vertical coordinate of the lower edge of the circumscribed rectangle as the vertical coordinate of the lower endpoint of the vertical line corresponding to the circumscribed rectangle. For example, the vertical coordinate of the upper edge of the circumscribed rectangle is y up , the lower edge has a vertical coordinate of y down , the horizontal coordinate of the left edge is x left , the horizontal coordinate of the right edge is x right , then the upper endpoint of the vertical line corresponding to the circumscribed rectangle is The lower endpoint is

[0128] In the method provided in the embodiment of the present application, after detecting the horizontal lines and vertical lines of the text image to be located after image preprocessing, a certain blank gap needs to be left for text area segmentation. Therefore, based on each text area, the horizontal lines above and / or below the text area are copied, and the vertical lines to the left and / or right of the text area are copied, and the copied horizontal lines are moved up or down, and the copied vertical lines are moved left or right.

[0129] In the method provided in the embodiment of the present application, each horizontal line and each vertical line is composed into a straight line frame of the text image to be located, and the composed straight line frame divides the text image to be located into multiple rectangles.

[0130] S108: Determine the text line area in the text image to be located according to the coordinate values ​​of the rectangle.

[0131] Based on the coordinate values ​​of each rectangle, the text line area in the text image to be located is determined. That is to say, the coordinate values ​​of each rectangle can determine a rectangle, and the determined rectangle corresponds to a text line area in the text image to be located. The coordinate values ​​of all rectangles can determine all text line areas in the text image to be located.

[0132] S109, if the image type of the text image to be located is a complex background layout type, the text image to be located is input into a pre-built single word recognition model to obtain the coordinate prediction value and confidence of the single word box corresponding to each single word in the text image to be located.

[0133] In the method provided in the embodiment of the present application, a single-word recognition model is pre-constructed. For the construction process of the single-word recognition model, please refer to the prior art and will not be described in detail here.

[0134] If the image type of the text image to be located is a complex background layout type, the text image to be located is input into a pre-built single-word recognition model to obtain the coordinate prediction value and confidence of the single-word box corresponding to each single word in the text image to be located output by the single-word recognition model. Optionally, the confidence range is between 0 and 1, and the larger the value, the higher the confidence.

[0135] S110 , determining a plurality of target single-word frames from the single-word frames based on the confidence of each single-word frame.

[0136] According to the confidence of each single word frame, multiple target single word frames are determined from the single word frame. For details, see Figure 5 , based on the confidence of each single word frame, the process of determining multiple target single word frames from each single word frame includes:

[0137] S501: For each single-word frame, if the confidence of the single-word frame is not less than a preset confidence threshold, the single-word frame is determined as an initial single-word frame.

[0138] The single-word frame whose confidence is not less than a preset confidence threshold among the single-word frames is determined as the initial single-word frame.

[0139] S502, grouping the initial single-word frames into a single-word frame set.

[0140] S503, selecting a first single-word frame from the current single-word frame set; the first single-word frame is an initial single-word frame with the highest confidence among the initial single-word frames included in the current single-word frame set.

[0141] S504: For each remaining initial single-character frame in the single-character frame set, calculate an area overlap ratio between the initial single-character frame and the first single-character frame.

[0142] For each remaining initial single-character frame in the single-character frame set, the area overlap rate of the initial single-character frame and the first single-character frame is calculated by calculating the area of ​​intersection and the combined area of ​​the initial single-character frame and the first single-character frame, that is, the area of ​​intersection is divided by the combined area to obtain the area overlap rate.

[0143] S505 . For each remaining initial single-character frame in the single-character frame set, determine whether an area overlap ratio between the initial single-character frame and the first single-character frame is greater than a preset overlap threshold.

[0144] For each remaining initial single-character frame in the single-character frame set, based on the calculated area overlap ratio between the initial single-character frame and the first single-character frame, determine whether the area overlap ratio is greater than a preset overlap threshold; if the area overlap ratio is greater than the preset overlap threshold, execute step S506; if the area overlap ratio is not greater than the preset overlap threshold, execute step S507.

[0145] S506: Delete the initial single-word frame from the single-word frame set.

[0146] For each remaining initial single-character frame in the single-character frame set, if the area overlap ratio is greater than a preset overlap threshold, the initial single-character frame is deleted from the single-character frame set, and step S507 is executed.

[0147] S507: Determine whether there is an initial single-character frame in the single-character frame set whose area overlap ratio is not calculated with the first single-character frame.

[0148] It is determined whether there is an initial single-character frame in the single-character frame set whose area overlap ratio is not calculated with the first single-character frame. If so, the process returns to step S505; if not, the process proceeds to step S508.

[0149] S508: Determine the first single-word frame as the target single-word frame.

[0150] S509: Determine whether the current single-word frame set is an empty set.

[0151] Determine whether the current single-character frame set is an empty set. If so, end the process directly. If not, return to step S503.

[0152] Optionally, in the method provided in the embodiment of the present application, the target single-word box whose height is not greater than a preset threshold is deleted.

[0153] S111, merging target single-character frames adjacent to each other in the horizontal direction to obtain multiple text line connected regions.

[0154] Merge the target single-word frames adjacent to each other in the horizontal direction to obtain multiple text line connected regions. The specific process includes:

[0155] According to the horizontal coordinate of the upper left corner of each target single word frame, the target single word frames are sorted in a preset order to obtain a single word frame sequence;

[0156] Determine whether the distance between the upper left corner horizontal coordinates of two adjacent target single-word frames in the single-word frame sequence is greater than a preset first threshold, and segment the middle of the two target single-word frames corresponding to whether the distance between the upper left corner horizontal coordinates is greater than the preset first threshold to obtain multiple single-word frame sequences;

[0157] The left boundary of the first target single-word frame in each single-word frame sequence is used as the left boundary of the text line connected area corresponding to the single-word frame sequence, the right boundary of the last target single-word frame in each single-word frame sequence is used as the right boundary of the text line connected area corresponding to the single-word frame sequence, the minimum value of the upper boundary of each single-word frame sequence is used as the upper boundary of the text line connected area corresponding to the single-word frame sequence, and the maximum value of the lower boundary of each single-word frame sequence is used as the lower boundary of the text line connected area corresponding to the single-word frame sequence.

[0158] In the method provided in the embodiment of the present application, the target single-word frames are arranged in a preset order according to the horizontal coordinate of the upper left corner of each target single-word frame to obtain a single-word frame sequence. Optionally, the preset order can be in the order of the horizontal coordinate from small to large, and it is judged whether the distance between the upper left corner horizontal coordinates of two adjacent target single-word frames in the single-word frame sequence is greater than a preset first threshold value. If it is greater, the two target single-word frames are cut to obtain multiple single-word frame sequences. That is to say, if there are multiple groups of adjacent two single-word frames in the single-word frame sequence, the distance between the upper left corner horizontal coordinates of the left key is greater than the preset first threshold value, the single-word frame sequence is cut. The columns are divided into multiple single-word frame sequences. For example, if there are 5 groups of adjacent two single-word frames whose upper left corner horizontal coordinate left key distance is greater than a preset first threshold, they are finally divided into 6 groups of single-word frame sequences. Each group of single-word frame sequences corresponds to a text line connected area. The upper boundary of the text line connected area is determined according to the minimum value of the upper boundary of the corresponding single-word frame sequence, the lower boundary is determined according to the maximum value of the lower boundary of the corresponding single-word frame sequence, the left boundary is determined according to the left boundary of the first target single-word frame in the corresponding single-word frame sequence, and the right boundary is determined according to the right boundary of the last target single-word frame in the corresponding single-word frame sequence.

[0159] S112, performing horizontal line detection and vertical line detection on the text image to be located.

[0160] The specific implementation process of step S112 is as described in step S107 and will not be repeated here.

[0161] S113 , determining a text line region in the text image to be located according to each text line connected region and the detected horizontal lines and vertical lines.

[0162] After determining the straight line frame and each text line connected area in the text image to be located, the text line connected area is divided by the straight line frame, that is, the detected horizontal lines and vertical lines constitute the straight line frame of the text image to be located, and the text image to be located is divided into multiple areas, thereby dividing the text belonging to different areas in the text connected area, thereby obtaining multiple final text line connected areas, each of which can determine a rectangle, and the determined rectangle corresponds to a text line area in the text image to be located, and the coordinate values ​​of all rectangles can determine all text line areas in the text image to be located.

[0163] The image text area positioning method provided by the embodiment of the present application adopts different image text area positioning strategies for different types of text images. For the text image of the pure text type, the text image is expanded so that the adjacent characters are connected into a text line connected area, and then the circumscribed rectangle of the text line connected area is determined to realize the positioning of the area where the text exists in the text image; for the text image of the text straight line staggered type, the area where the text exists in the text image is positioned by detecting the straight line frame in the text image, and for the text image of the complex background layout type, the single word frame corresponding to each single word in the text image is identified based on the single word recognition model, and the single word frame is merged into a text line connected area, and the straight line frame in the text image is detected, and the straight line frame and the text line connected area are used to realize the positioning of the area where the text exists in the text image. The image text area positioning method provided by the embodiment of the present application is adopted, and the upper, lower, left and right edge positions of each text line in the text image are accurately positioned by identifying the circumscribed rectangle of the straight line frame and / or the connected area in the text image, and different image text area positioning strategies are adopted for different types of text images, so that the image text area positioning of each type of text image has universality.

[0164] and Figure 1 Corresponding to the method described above, the embodiment of the present application further provides an image text area positioning device for Figure 1 The specific implementation of the method is shown in the following diagram: Figure 6 As shown, specifically including:

[0165] An acquisition unit 601 is used to acquire a text image to be located and determine the image type of the text image to be located; the image type includes a pure text type, a text straight line staggered type, or a complex background layout type;

[0166] The first positioning unit 602 is used for, if the image type of the text image to be positioned is a pure text type, performing image preprocessing on the text image to be positioned, and performing dilation processing on the text image to be positioned after the image preprocessing to obtain a target text image, identifying each text line connected area in the target text image, determining the coordinate value of the circumscribed rectangle of each text line connected area, and determining the text line area in the text image to be positioned based on the coordinate value of the circumscribed rectangle of each text line connected area; wherein the pixel values ​​of adjacent pixel points in each text line connected area are the same;

[0167] The second positioning unit 603 is used for performing image preprocessing on the text image to be positioned if the image type of the text image to be positioned is a text straight line interlaced type, and performing horizontal line detection and vertical line detection on the text image to be positioned after the image preprocessing, and determining a plurality of rectangles based on each horizontal line and each vertical line obtained by the detection, and determining a text line area in the text image to be positioned according to the coordinate values ​​of the rectangles;

[0168] The third positioning unit 604 is used for inputting the text image to be positioned into a pre-constructed single-word recognition model if the image type of the text image to be positioned is a complex background layout type, obtaining the coordinate prediction value and confidence of the single-word frame corresponding to each single word in the text image to be positioned, determining a plurality of target single-word frames from each of the single-word frames based on the confidence of each of the single-word frames, merging the target single-word frames adjacent in the horizontal direction to obtain a plurality of text line connected areas, performing horizontal line detection and vertical line detection on the text image to be positioned, and determining the text line area in the text image to be positioned according to each of the text line connected areas and the detected horizontal lines and vertical lines.

[0169] The image text area positioning device provided by the embodiment of the present application adopts different image text area positioning strategies for different types of text images. For the text image of the pure text type, the text image is expanded so that the adjacent characters are connected into a text line connected area, and then the circumscribed rectangle of the text line connected area is determined to realize the positioning of the area where the text exists in the text image; for the text image of the text straight line staggered type, the area where the text exists in the text image is positioned by detecting the straight line frame in the text image, and for the text image of the complex background layout type, the single word frame corresponding to each single word in the text image is identified based on the single word recognition model, and the single word frame is merged into a text line connected area, and the straight line frame in the text image is detected, and the straight line frame and the text line connected area are used to realize the positioning of the area where the text exists in the text image. The image text area positioning device provided by the embodiment of the present application is adopted, and the upper, lower, left and right edge positions of each text line in the text image are accurately positioned by identifying the circumscribed rectangle of the straight line frame and / or the connected area in the text image, and different image text area positioning strategies are adopted for different types of text images, so that the image text area positioning of each type of text image has universality.

[0170] In one embodiment of the present application, based on the above solution, the first positioning unit 602 and the second positioning unit 603 are configured as follows:

[0171] A grayscale subunit, used for grayscale processing the image of the text to be located to obtain a grayscale image;

[0172] A filtering subunit, used for filtering the grayscale image to obtain a filtered image;

[0173] A binarization subunit, used for performing adaptive binarization processing on the filtered image to obtain a binarized image;

[0174] The inversion subunit is used to invert the pixel value of each pixel in the binary image.

[0175] In one embodiment of the present application, based on the above solution, the filtering subunit performs filtering processing on the grayscale image to obtain a filtered image, which is used for:

[0176] Slide the center of a preset filtering sliding window across each pixel in the grayscale image;

[0177] Each time the center of the filtering sliding window slides to a pixel point in the grayscale image, based on the noise type of the text image to be located, a preset filtering calculation formula corresponding to the noise type is selected, and based on the selected filtering calculation method, the filtering grayscale value within the current filtering sliding window is calculated, and the calculated filtering grayscale value is used as the pixel value of the pixel point.

[0178] In one embodiment of the present application, based on the above solution, the first positioning unit 602 performs dilation processing on the image to be positioned after image preprocessing to obtain a target text image, which is used for:

[0179] Based on the first sliding window, the text image to be located after image preprocessing is expanded; wherein the width of the first sliding window is determined according to the spacing between adjacent characters in the text image to be located, and the height of the first sliding window is determined according to the line spacing of text lines in the image to be located.

[0180] In one embodiment of the present application, based on the above solution, the second positioning unit 603 performs horizontal line detection on the image of the text to be positioned after image preprocessing, for:

[0181] Based on a preset second sliding window, the image of the text to be located is corroded after image preprocessing to obtain a first corroded image;

[0182] Based on a preset third sliding window, dilating the first eroded image to obtain a first dilated image;

[0183] Identify each horizontally connected region in the first dilated image, and determine a circumscribed rectangle of each horizontally connected region;

[0184] For each of the horizontally connected regions, the coordinates of two end points of a horizontal line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the horizontally connected region.

[0185] In one embodiment of the present application, based on the above solution, the second positioning unit 603 performs vertical line detection on the image of the text to be positioned after image preprocessing, for:

[0186] Based on a preset fourth sliding window, the image of the text to be located is corroded after image preprocessing to obtain a second corroded image;

[0187] Based on a preset fifth sliding window, dilating the second eroded image to obtain a second dilated image;

[0188] identifying each vertically connected region in the second dilated image, and determining a circumscribed rectangle of each vertically connected region;

[0189] For each of the vertically connected regions, the coordinates of two end points of a vertical line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the vertically connected region.

[0190] In one embodiment of the present application, based on the above solution, the third positioning unit 604 determines a plurality of target single-word frames from each single-word frame based on the confidence of each single-word frame, for:

[0191] For each of the single-word frames, if the confidence of the single-word frame is not less than a preset confidence threshold, the single-word frame is determined as an initial single-word frame;

[0192] Grouping the initial single-word frames into a single-word frame set;

[0193] Selecting a first single-word frame from the current single-word frame set; wherein the first single-word frame is an initial single-word frame with the largest confidence among the initial single-word frames included in the current single-word frame set;

[0194] For each remaining initial single-character frame in the single-character frame set, calculating an area overlap ratio between the initial single-character frame and the first single-character frame, and if the area overlap ratio is greater than a preset overlap threshold, deleting the initial single-character frame from the single-character frame set;

[0195] Determine the target single-word frame by using the first single-word frame, and determine whether the current single-word frame set is an empty set;

[0196] If the current single-character frame set is not an empty set, the process returns to the step of selecting the first single-character frame from the current single-character frame set until the current single-character frame set is an empty set.

[0197] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned image text area positioning method.

[0198] The present application also provides an electronic device, whose structural diagram is as follows: Figure 7 As shown, it specifically includes a memory 701 and one or more instructions 702, wherein the one or more instructions 702 are stored in the memory 701 and are configured to be executed by one or more processors 703 to perform the following operations:

[0199] Acquire a text image to be located, and determine the image type of the text image to be located; the image type includes a pure text type, a text straight line staggered type, or a complex background layout type;

[0200] If the image type of the to-be-located text image is a pure text type, the to-be-located text image is subjected to image preprocessing, and the to-be-located text image after image preprocessing is subjected to dilation processing to obtain a target text image, and each text line connected area in the target text image is identified, and the coordinate value of the circumscribed rectangle of each of the text line connected areas is determined, and based on the coordinate value of the circumscribed rectangle of each of the text line connected areas, the text line area in the to-be-located text image is determined; wherein the pixel values ​​of adjacent pixel points in each of the text line connected areas are the same;

[0201] If the image type of the to-be-located text image is a text straight line interlaced type, the to-be-located text image is subjected to image preprocessing, and horizontal line detection and vertical line detection are performed on the to-be-located text image after image preprocessing, and multiple rectangles are determined based on each horizontal line and each vertical line obtained by detection, and a text line area in the to-be-located text image is determined according to the coordinate values ​​of the rectangles;

[0202] If the image type of the text image to be located is a complex background layout type, the text image to be located is input into a pre-constructed single-word recognition model to obtain the coordinate prediction value and confidence of the single-word frame corresponding to each single word in the text image to be located, and based on the confidence of each single-word frame, multiple target single-word frames are determined from each of the single-word frames, and the target single-word frames adjacent in the horizontal direction are merged to obtain multiple text line connected areas, and horizontal line detection and vertical line detection are performed on the text image to be located. According to each of the text line connected areas and the detected horizontal lines and vertical lines, the text line area in the text image to be located is determined.

[0203] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0204] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0205] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0206] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0207] The above is a detailed introduction to an image text area positioning method and device, storage medium and electronic device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for locating text areas in an image, characterized in that: include: Acquire a text image to be located, and determine the image type of the text image to be located; the image type includes a pure text type, a text straight line staggered type, or a complex background layout type; If the image type of the to-be-located text image is a pure text type, the to-be-located text image is subjected to image preprocessing, and the to-be-located text image after image preprocessing is subjected to dilation processing to obtain a target text image, and each text line connected area in the target text image is identified, and the coordinate value of the circumscribed rectangle of each of the text line connected areas is determined, and based on the coordinate value of the circumscribed rectangle of each of the text line connected areas, the text line area in the to-be-located text image is determined; wherein the pixel values ​​of adjacent pixel points in each of the text line connected areas are the same; If the image type of the to-be-located text image is a text straight line interlaced type, the to-be-located text image is subjected to image preprocessing, and horizontal line detection and vertical line detection are performed on the to-be-located text image after image preprocessing, and multiple rectangles are determined based on each horizontal line and each vertical line obtained by detection, and a text line area in the to-be-located text image is determined according to the coordinate values ​​of the rectangles; If the image type of the text image to be positioned is a complex background layout type, the text image to be positioned is input into a pre-constructed single-word recognition model to obtain a coordinate prediction value and a confidence level of a single-word frame corresponding to each single word in the text image to be positioned, and based on the confidence level of each single-word frame, a plurality of target single-word frames are determined from each of the single-word frames, and the target single-word frames adjacent in the horizontal direction are merged to obtain a plurality of text line connected areas, and horizontal line detection and vertical line detection are performed on the text image to be positioned, and a text line area in the text image to be positioned is determined according to each of the text line connected areas and the detected horizontal lines and vertical lines; Wherein, determining a plurality of target single-word frames from each of the single-word frames based on the confidence of each of the single-word frames comprises: For each of the single-word frames, if the confidence of the single-word frame is not less than a preset confidence threshold, the single-word frame is determined as an initial single-word frame; Grouping the initial single-word frames into a single-word frame set; Selecting a first single-word frame from the current single-word frame set; wherein the first single-word frame is an initial single-word frame with the largest confidence among the initial single-word frames included in the current single-word frame set; For each remaining initial single-character frame in the single-character frame set, calculating an area overlap ratio between the initial single-character frame and the first single-character frame, and if the area overlap ratio is greater than a preset overlap threshold, deleting the initial single-character frame from the single-character frame set; Determine the target single-word frame by using the first single-word frame, and determine whether the current single-word frame set is an empty set; If the current single-character frame set is not an empty set, the process returns to the step of selecting the first single-character frame from the current single-character frame set until the current single-character frame set is an empty set.

2. The method according to claim 1, characterized in that The performing image preprocessing on the to-be-located text image comprises: Performing grayscale processing on the image of the text to be located to obtain a grayscale image; Performing filtering on the grayscale image to obtain a filtered image; Performing adaptive binarization processing on the filtered image to obtain a binarized image; The pixel value of each pixel in the binary image is inverted.

3. The method according to claim 2, characterized in that The filtering process is performed on the grayscale image to obtain a filtered image, including: Slide the center of a preset filtering sliding window across each pixel in the grayscale image; Every time the center of the filtering sliding window slides to a pixel point in the grayscale image, based on the noise type of the text image to be located, a preset filtering calculation formula corresponding to the noise type is selected, and based on the selected filtering calculation method, the filtering grayscale value in the current filtering sliding window is calculated, and the calculated filtering grayscale value is used as the pixel value of the pixel point.

4. The method according to claim 3, characterized in that The step of performing expansion processing on the to-be-located text image after image preprocessing to obtain a target text image comprises: Based on the first sliding window, the text image to be located after image preprocessing is expanded; wherein the width of the first sliding window is determined according to the spacing between adjacent characters in the text image to be located, and the height of the first sliding window is determined according to the line spacing of text lines in the image to be located.

5. The method according to claim 3, characterized in that: The step of performing horizontal line detection on the image of the text to be located after image preprocessing comprises: Based on a preset second sliding window, the image of the text to be located is corroded after image preprocessing to obtain a first corroded image; Based on a preset third sliding window, dilating the first eroded image to obtain a first dilated image; Identify each horizontally connected region in the first dilated image, and determine a circumscribed rectangle of each horizontally connected region; For each of the horizontally connected regions, the coordinates of two end points of a horizontal line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the horizontally connected region.

6. The method according to claim 3, characterized in that The step of performing vertical line detection on the image of the text to be located after image preprocessing comprises: Based on a preset fourth sliding window, the image of the text to be located is corroded after image preprocessing to obtain a second corroded image; Based on a preset fifth sliding window, dilating the second eroded image to obtain a second dilated image; identifying each vertically connected region in the second dilated image, and determining a circumscribed rectangle of each vertically connected region; For each of the vertically connected regions, the coordinates of two end points of a vertical line corresponding to the circumscribed rectangle are calculated according to the coordinates of the circumscribed rectangle of the vertically connected region.

7. An image text area positioning device, characterized in that: include: An acquisition unit, used for acquiring a text image to be located, and determining an image type of the text image to be located; the image type includes a pure text type, a text straight line staggered type, or a complex background layout type; A first positioning unit is used for, if the image type of the text image to be positioned is a pure text type, performing image preprocessing on the text image to be positioned, and performing dilation processing on the text image to be positioned after the image preprocessing to obtain a target text image, identifying each text line connected area in the target text image, determining the coordinate value of the circumscribed rectangle of each text line connected area, and determining the text line area in the text image to be positioned based on the coordinate value of the circumscribed rectangle of each text line connected area; wherein the pixel values ​​of adjacent pixel points in each text line connected area are the same; A second positioning unit is used for, if the image type of the text image to be positioned is a text straight line interlaced type, performing image preprocessing on the text image to be positioned, and performing horizontal line detection and vertical line detection on the text image to be positioned after the image preprocessing, determining a plurality of rectangles based on each horizontal line and each vertical line obtained by the detection, and determining a text line area in the text image to be positioned according to the coordinate values ​​of the rectangles; A third positioning unit is used for, if the image type of the text image to be positioned is a complex background layout type, inputting the text image to be positioned into a pre-constructed single word recognition model, obtaining a coordinate prediction value and a confidence level of a single word frame corresponding to each single word in the text image to be positioned, determining a plurality of target single word frames from each of the single word frames based on the confidence level of each of the single word frames, merging the target single word frames adjacent in the horizontal direction to obtain a plurality of text line connected areas, performing horizontal line detection and vertical line detection on the text image to be positioned, and determining a text line area in the text image to be positioned according to each of the text line connected areas and the detected horizontal lines and vertical lines; Wherein, based on the confidence of each of the single-word frames, a third positioning unit of a plurality of target single-word frames is determined from each of the single-word frames, specifically used for: For each of the single-word frames, if the confidence of the single-word frame is not less than a preset confidence threshold, the single-word frame is determined as an initial single-word frame; the initial single-word frames are grouped into a single-word frame set; a first single-word frame is selected from the current single-word frame set; the first single-word frame is the initial single-word frame with the largest confidence among the initial single-word frames contained in the current single-word frame set; for each of the remaining initial single-word frames in the single-word frame set, an area overlap ratio between the initial single-word frame and the first single-word frame is calculated, and if the area overlap ratio is greater than a preset overlap threshold, the initial single-word frame is deleted from the single-word frame set; the first single-word frame is determined as a target single-word frame, and it is determined whether the current single-word frame set is an empty set; if the current single-word frame set is not an empty set, the step of selecting the first single-word frame from the current single-word frame set is returned to be executed until the current single-word frame set is an empty set.

8. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the image text area positioning method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The system comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to execute the image text area positioning method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Method for quickly calculating layout structure of document image

    CN102073862A

  • Identification method of form type

    CN103034848A

  • Deep learning and OCR-based bill classification and bill field extraction method

    CN107633239A

  • An image normalization method and device based on table characteristics

    CN109948135A

  • Method for extracting structured information from real estate certificate image

    CN111460927A