Picture character recognition method and system based on layout optimization voting

The method optimizes text recognition by adjusting image text layout and voting on multiple recognition results, addressing the limitations of current systems in handling multi-line and irregular text images, enhancing recognition accuracy.

CN119068499AActive Publication Date: 2024-12-03CENT SOUTH UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411193571.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-03
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

Current optical character recognition systems face challenges in accurately recognizing specific text documents due to poor recognition accuracy, especially with multi-line and irregularly distributed text images, and insufficient attention is given to the impact of text layout on recognition effectiveness.

Method used

A method and system that optimize text recognition by adjusting the layout of text in images through layout optimization voting, involving preprocessing steps like image enhancement, text line segmentation, and voting on multiple layout recognition results to enhance accuracy.

Benefits of technology

Significantly improves recognition accuracy by standardizing text layout, enhancing text information, and using character size to simulate spacing, resulting in more accurate character and line spacing adjustments, thereby improving overall recognition performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068499A_ABST
    Figure CN119068499A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of picture and character recognition, and particularly discloses a picture and character recognition method and system based on layout optimization voting, and the method comprises the following steps: step 100, inputting a text image file; step 200, preprocessing the input image; step 300, adjusting the layout of the image characters; step 400, multi-layout line image character recognition and multi-layout recognition result voting; and step 500, outputting a result list, and ending. According to the method, the character size is estimated according to the row height in combination with the expression characteristic of the square of the dot matrix Chinese character, the character layout is adjusted by simulating the character spacing and the row spacing according to the character size, and finally, voting is performed on various character layout recognition results to obtain a final result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of picture text recognition, and in particular to a picture text recognition method based on layout optimization voting, and also to a picture text recognition system based on layout optimization voting. Background Art

[0002] The recognition accuracy of currently commonly used open source optical character recognition systems has reached a very high level when facing general text documents. However, when faced with text recognition tasks for specific text documents, the recognition accuracy often fails to meet expectations.

[0003] On the one hand, the traditional image pattern matching algorithm has poor recognition accuracy. On the other hand, it is often unable to accurately divide dense text images into character levels, which will also cause inaccurate text recognition. The commonly used DB-CRNN two-step text recognition system is often unable to accurately locate all text positions when faced with multi-line and irregularly distributed text images, resulting in a large number of missed recognitions.

[0004] At present, the optimization of text recognition systems from the perspective of preprocessing usually focuses on improving image quality and enhancing the text information in the image. However, in actual recognition tasks, after operations such as image enhancement and denoising, the recognition effect is still not ideal. At present, little attention is paid to the impact of text layout in the image on the recognition effect. Summary of the invention

[0005] In view of this, the present invention provides a method and system for image text recognition based on layout optimization voting, so as to optimize the optical character recognition from the perspective of improving the input image preprocessing by adjusting the layout of the text in the image.

[0006] In order to achieve the above object, the basic scheme of the present invention provides a method for image text recognition based on layout optimization voting, comprising the following steps:

[0007] Step 100, input text image file;

[0008] Step 200, input image preprocessing;

[0009] Step 300, image and text layout adjustment;

[0010] Step 400, multi-layout line image text recognition and multi-layout recognition result voting;

[0011] Step 500, output the result list Result, and end.

[0012] In one possible design, step 200 specifically includes:

[0013] Step 201, read the image file and obtain the suffix suffix;

[0014] Step 202, determine suffix, if suffix = ".tif", do not process, otherwise, modify the file format so that suffix = ".tif";

[0015] Step 203, using OpenCV to read the tif format image file to obtain the image Image;

[0016] Step 204, using OpenCV to perform grayscale conversion on Image to obtain a grayscale image Image_gray, and performing local mean method adaptive binarization on Image_gray to obtain a binarized image Image_bianry;

[0017] Step 205, using OpenCV to set a 2*2 convolution kernel for Image_bianry, and perform an erosion, dilation, and erosion operation on Image_bianry to obtain the result Image_ede, and Image=Image_ede; the input image preprocessing is completed, and step 300 is entered.

[0018] In one possible design, step 300 includes,

[0019] Step 301, read the pre-processed image Image, read the Image size as h in height and w in width,

[0020] Step 302, horizontally project the Image, and record the sum of the pixel values ​​of each row of the Image pixel matrix to obtain a horizontal projection vector Hor_proj;

[0021] Step 303, read the horizontal projection vector Hor_proj, and record the position si where the value changes from 0 and the position ei where the value changes to 0 from top to bottom, and obtain the top and bottom positions of the row rowi = [si, ei], i = 1, 2, 3, ..., n, and all row positions row_all = [row1, row2, row3, ..., rown];

[0022] Step 304, traverse row_all, initialize the list row_width, calculate the absolute value of the difference between two elements in rowi, add it to the list row_width, and take the arithmetic mean width_mean of all elements in row_width;

[0023] Step 305, traverse row_all and divide the image Image into

[0024] Image_rowi=Image[rowi[0]:rowi[1],0:w], get the focused row image set

[0025] Image_row_set=[Image_row1,Image_row2,…,Image_rown];

[0026] Step 306, traverse Image_row_set, perform vertical projection on Image_rowi, and record the sum of the pixel values ​​of each column in the image pixel matrix to obtain the vertical projection vector Ver_proji;

[0027] Step 307, read the vertical projection vector Ver_proji corresponding to the row image Image_rowi in sequence, and record the position si where the value changes from 0 and the position ei where the value changes to 0 in sequence from left to right, and obtain the left column position and the right column position coli = [si, ei], and all column positions col_alli = [col1, col2, col3, ..., coln];

[0028] Step 308, read the column position list col_alli corresponding to the row image Image_rowi in turn, cluster the elements in the list col_alli based on the distance threshold, traverse col_alli, if |colj[0]-colj-1[1]|≤t, then do not process, otherwise merge colj and colj-1 to obtain [colj-1[0],colj[1]], j=1, 2, 3, ..., n; process to obtain the corresponding merged column position list col_merge_alli=[col_merge1, col_merge2, ..., col_mergem];

[0029] Step 309, traverse all row images Image_rowi, read the Image_rowi size rowi_h and rowi_w corresponding to the merged column position list co_merge_alli, and split Image_rowi according to the column position recorded in col_merge_alli.

[0030] rowi_colj=Image_rowi[0:rowi_h,col_mergej[0]:col_mergej[1]], get the corresponding column image set Image_rowi_col_set={rowi_col1, rowi_col2, …, rowi_colm};

[0031] Step 310, initialize the sub-image width list sub_col_width, traverse all the column image sets Image_rowi_col_set corresponding to the row images Image_rowi, read the sub-image size width rowi_colj_w, add sub_col_width and the number of sub-images rowi_col_cnt;

[0032] Step 311, sum all elements in the sub_col_width corresponding to Image_rowi to obtain w_sumi, initialize the size of the blank canvas Image_rowi_adjust to rowi_h*(w_sumi+int(1 / 3)*w_mean*(rowi_col_cnt-1)), i=1, 2, ..., m, and integrate the sub-images in the corresponding column image set Image_rowi_col_set into Image_rowi_adjust in sequence, with the sub-image spacing being int(1 / 3)*w_mean; process all row images to obtain the adjusted row image set Image_row_adjust_set=

[0033] {Image_row1_adjust,…Image_rowm_adjust} collection;

[0034] Step 312, traverse the adjusted row image set Image_row_adjust_set, use OpenCV to adjust the background size of a single row image Image_rowi_adjust, and fill the four sides of the row image with blanks using the corresponding row height rowi_h as the step size, and the n step size filling size is (rowi_h+2*n*rowi_h)*(rowi_w+2*rowi_w), and each row image is filled with 5 unit steps to obtain a set of adjusted image sets

[0035] rowi_adjust_bk_set={rowi_h,rowi_2h,rowi_3h,rowi_4h,rowi_5h};Perform the same processing on all row images of all images;After image and text layout adjustment is completed, go to step 400.

[0036] In one possible design, the step 400 includes,

[0037] Step 401, initialize the text row multi-layout recognition result list rowi_result, and the row image multi-layout image set rowi_adjust_bk_set = {rowi_h, rowi_2h, rowi_3h, rowi_4h, rowi_5h};

[0038] Step 402, traverse the multi-layout image set rowi_adjust_bk_set, input all images in the set into the text recognition engine for recognition, obtain recognition results, and add them to the recognition result list rowi_result,

[0039] Final recognition result list rowi_result = [rowi_result_h, rowi_result_2h, rowi_result_3h, rowi_result_4h, rowi_result_5h];

[0040] Step 403, vote on the results in rowi_result, and the results with empty recognition results are not included in the vote. The recognition result with the highest frequency of results is taken as the final recognition result of row rowi, and all row images are recognized and voted on, and added to the result list Result in sequence. In a possible design, the text image file in step 100 includes jpg, png or tif.

[0041] In a possible design, in step S308, the distance threshold is t=1*width_mean, where width_mean is the average line height of the image text.

[0042] The present invention also provides a picture and text recognition device based on layout optimization voting, including a memory, a control processor and a computer program stored in the memory and executable on the control processor, wherein the control processor executes the program to implement the picture and text recognition method based on layout optimization voting as described above.

[0043] The present invention also provides a control system, including the above-mentioned image and text recognition device based on layout optimization voting.

[0044] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the above-mentioned image text recognition method based on layout optimization voting.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention combines the square representation characteristics of dot matrix Chinese characters, estimates the character size by line height, and uses the character size to simulate the character spacing and line spacing to adjust the text layout, and finally votes on multiple text layout recognition results to obtain the final result.

[0047] In addition to the recognition system itself, the input image of the present invention also affects the accuracy of the recognition results of the optical character recognition system. Regularly arranged and clear images are conducive to the recognition system's accurate recognition of characters in the image. The present invention performs text line segmentation on the image through Hybrid Segments on the basis of grayscale and binarization, corrosion and expansion of the input image to enhance text information, and uses the text line height to approximate the character width to adjust the character spacing and line spacing of the characters in the image, so that the image text is arranged in a standardized manner, and by adjusting the background size of the text line image, the purpose of adjusting the layout of the text in the image is achieved, a group of text images with different layouts are generated, and the images with different layouts are input into the recognition system, the recognition results are voted, and the result with the highest score is used as the final recognition result, and the recognition accuracy can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A logical block diagram of a method and system for image text recognition based on layout optimization voting proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0051] Disadvantages of existing technology:

[0052] (1) Traditional image pattern matching algorithms have poor recognition accuracy and are often unable to accurately segment dense text images at the character level, which can also cause inaccurate text recognition. The commonly used DB-CRNN two-step text recognition system is often unable to accurately locate all text positions when faced with multi-line and irregularly distributed text images, resulting in a large number of missed recognitions.

[0053] (2) At present, the optimization of text recognition systems from the perspective of preprocessing usually focuses on improving image quality and enhancing the text information in the image. However, in actual recognition tasks, after operations such as image enhancement and denoising, the recognition effect is still not ideal. At present, little attention has been paid to the impact of text layout in the image on the recognition effect.

[0054] Therefore, after performing image enhancement processing such as tilt correction, grayscale and binarization, and filtering on the input image, the present invention focuses the text line sequence image through Hybrid Segment, and adjusts the text layout model in the image by simulating the character spacing and line spacing through the character size, and then inputs it into the text recognition system to improve the recognition accuracy and completeness.

[0055] like Figure 1 As shown, a method for picture text recognition based on layout optimization voting of the present invention comprises the following steps:

[0056] Step 100, inputting a text image file; specifically including:

[0057] Step 101, input text image file (including image formats such as .jpg / .png / .tif).

[0058] Step 102, output description: The output is the character recognition result in the text image, stored in lines, and the result list Result.

[0059] Step 200, input image preprocessing; specifically including:

[0060] Step 201, read the image file and obtain the suffix suffix;

[0061] Step 202, determine suffix, if suffix = ".tif", do not process, otherwise, modify the file format so that suffix = ".tif"; go to step 203;

[0062] Step 203, using OpenCV to read the tif format image file to obtain the image Image; then proceed to step 204;

[0063] Step 204, using OpenCV to perform grayscale conversion on Image to obtain a grayscale image Image_gray, performing local mean method adaptive binarization on Image_gray to obtain a binarized image Image_bianry, and then proceeding to step 205;

[0064] Step 205, using OpenCV to set a 2*2 convolution kernel for Image_bianry, and perform an erosion, dilation, and erosion operation on Image_bianry to obtain the result Image_ede, and Image=Image_ede; the input image preprocessing is completed, and step 300 is entered.

[0065] Step 300, image text layout adjustment; the purpose of image text layout adjustment is to adjust the character spacing and line spacing of the text in the image, so that the text recognition system can extract text features for accurate recognition. Its main steps include image text line focusing based on Hybrid Segment and text layout adjustment. The specific process is as follows: including,

[0066] Step 301, read the pre-processed image Image, read the Image size as h in height and w in width, and go to step 302;

[0067] Step 302, horizontally project the Image, and record the sum of the pixel values ​​of each row of the Image pixel matrix to obtain the horizontal projection vector Hor_proj, and then proceed to step 303;

[0068] Step 303, read the horizontal projection vector Hor_proj, and record the position si where the value changes from 0 and the position ei where the value changes to 0 from top to bottom, and obtain the top and bottom positions of the row rowi = [si, ei], i = 1, 2, 3, ..., n, and all row positions row_all = [row1, row2, row3, ..., rown] (i = 1, 2, 3, ..., n); proceed to step 304;

[0069] Step 304, traverse row_all, initialize the list row_width, calculate the absolute value of the difference between two elements in rowi, add it to the list row_width, and take the arithmetic mean width_mean of all elements in row_width; proceed to step 305;

[0070] Step 305, traverse row_all and divide the image Image into

[0071] Image_rowi=Image[rowi[0]:rowi[1],0:w], get the focused row image set

[0072] Image_row_set = [Image_row1, Image_row2, ..., Image_rown]; Go to step 306;

[0073] Step 306, traverse Image_row_set, perform vertical projection on Image_rowi, and record the sum of the pixel values ​​of each column in the image pixel matrix to obtain the vertical projection vector Ver_proji; proceed to step 307;

[0074] Step 307, read the vertical projection vector Ver_proji corresponding to the row image Image_rowi in sequence, and record the position si where the value changes from 0 and the position ei where the value changes to 0 in sequence from left to right, and obtain the left and right column positions coli = [si, ei], and all column positions col_alli = [col1, col2, col3, ..., coln]; go to step 308;

[0075] Step 308, read the column position list col_alli corresponding to the row image Image_rowi in turn, cluster the elements in the list col_alli based on the distance threshold, traverse col_alli, if |colj[0]-colj-1[1]|≤t, then do not process, otherwise merge colj and colj-1 to obtain [colj-1[0],colj[1]], j=1, 2, 3, ..., n; process to obtain the corresponding merged column position list col_merge_alli=[col_merge1, col_merge2, ..., col_mergem]; then go to step 309;

[0076] Step 309, traverse all row images Image_rowi, read the Image_rowi size rowi_h and rowi_w corresponding to the merged column position list co_merge_alli, and split Image_rowi according to the column position recorded in col_merge_alli.

[0077] rowi_colj=Image_rowi[0:rowi_h,col_mergej[0]:col_mergej[1]], j=1, 2, ..., m, obtain the corresponding column image set Image_rowi_col_set={rowi_col1, rowi_col2, ..., rowi_colm}; go to step 310;

[0078] Step 310, initialize the sub-image width list sub_col_width, traverse all the row images Image_rowi corresponding to the column image set Image_rowi_col_set, read the sub-image size width rowi_colj_w, add sub_col_width and the number of sub-images rowi_col_cnt; go to step 311;

[0079] Step 311, sum all elements in the sub_col_width corresponding to Image_rowi to obtain w_sumi, initialize the size of the blank canvas Image_rowi_adjust to rowi_h*(w_sumi+int(1 / 3)*w_mean*(rowi_col_cnt-1)), i=1, 2, ..., m, and integrate the sub-images in the corresponding column image set Image_rowi_col_set into Image_rowi_adjust in sequence, with the sub-image spacing being int(1 / 3)*w_mean; process all row images to obtain the adjusted row image set Image_row_adjust_set=

[0080] {Image_row1_adjust,,Image_rowm_adjust} set; go to step 312;

[0081] Step 312, traverse the adjusted row image set Image_row_adjust_set, use OpenCV to adjust the background size of a single row image Image_rowi_adjust, and its size is rowi_h*rowi_w. Then, fill the four sides of the row image with blanks using the corresponding row height rowi_h as the step size, and the filling size of n steps is (rowi_h+2*n*rowi_h)*(rowi_w+2*rowi_w). Each row image is filled with 5 unit steps, and a set of adjusted image set rowi_adjust_bk_set={rowi_h,rowi_2h,rowi_3h,rowi_4h,rowi_5h} is obtained; the same processing is performed on all row images of all images; the image and text layout adjustment is completed, and step 400 is entered.

[0082] Step 400, multi-layout line image text recognition and multi-layout recognition result voting; including,

[0083] Step 401, initialize the text row multi-layout recognition result list rowi_result, the row image multi-layout image set rowi_adjust_bk_set={rowi_h,rowi_2h,rowi_3h,rowi_4h,rowi_5h}; proceed to step 402;

[0084] Step 402, traverse the multi-layout image set rowi_adjust_bk_set, input all images in the set into the text recognition engine for recognition, obtain recognition results, and add them to the recognition result list rowi_result,

[0085] Final recognition result list rowi_result = [rowi_result_h, rowi_result_2h, rowi_result_3h, rowi_result_4h, rowi_result_5h]; Go to step 403;

[0086] Step 403, vote on the results in rowi_result, and the results with empty recognition results are not included in the vote. The recognition result with the highest frequency of occurrence is taken as the final recognition result of row rowi. Recognition and result voting are performed on all row images, and they are added to the result list Result in sequence, and then go to step 500.

[0087] Step 500, result list Result, end. The text image layout can be understood as the position and pattern of the distribution of the text line sequence in the image, which can be summarized as line spacing and character spacing. For a general multi-language text recognition system, an image that has undergone image cleaning and has appropriate and regular text line spacing and character spacing is more conducive to the text recognition system to accurately locate the text area, but it is difficult to accurately determine the appropriate size of the character spacing and line spacing. In response to the above problems, the present invention combines the representation characteristics of the square of dot matrix Chinese characters, estimates the character size by line height, and uses the character size to simulate the character spacing and line spacing to adjust the text layout, and finally votes on multiple text layout recognition results to obtain the final result.

[0088] The present invention also provides a picture and text recognition device based on layout optimization voting, including a memory, a control processor and a computer program stored in the memory and executable on the control processor, wherein the control processor executes the program to implement the picture and text recognition method based on layout optimization voting as described above.

[0089] The present invention also provides a control system, including the above-mentioned image and text recognition device based on layout optimization voting.

[0090] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the above-mentioned image text recognition method based on layout optimization voting.

[0091] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in different orders and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but understandable to those skilled in the art. It will be further appreciated by those skilled in the art that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly explain this interchangeability of hardware and software, various illustrative components, frames, modules, circuits, and steps are generally described in the form of their functionality above. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technical staff can implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention. The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, a battery compartment control board, a micro-battery compartment control board, or a state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components. In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on a computer-readable medium or transmitted therefrom as one or more instructions or codes.Computer-readable media include both computer storage media and communication media, including any media that facilitates a computer program to be transferred from one place to another. Storage media can be any available media that can be accessed by a computer. As an example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other media that can be used to carry or store the agreed program code in the form of instructions or data structures and can be accessed by a computer. Any connection is also properly referred to as computer-readable media. For example, if software is transmitted from a web site, a central control computer, or other remote sources using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0092] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0093] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for image text recognition based on layout optimization voting, characterized in that: The following steps are involved: Step 100, input text image file; Step 200, input image preprocessing; Step 300, image and text layout adjustment; Step 400, multi-layout line image text recognition and multi-layout recognition result voting; Step 500, output the result list Result, and end.

2. According to the method of picture text recognition based on layout optimization voting according to claim 1, it is characterized in that: Step 200 specifically includes: Step 201, read the image file and obtain the suffix suffix; Step 202, determine suffix, if suffix = ".tif" then do not process, otherwise modify the file format so that suffix = ".tif"; Step 203, using OpenCV to read the tif format image file to obtain the image Image; Step 204, using OpenCV to perform grayscale conversion on Image to obtain a grayscale image Image_gray, and performing local mean method adaptive binarization on Image_gray to obtain a binarized image Image_bianry; Step 205, using OpenCV to set a 2*2 convolution kernel for Image_bianry, and perform an erosion, dilation, and erosion operation on Image_bianry to obtain the result Image_ede, and Image=Image_ede; the input image preprocessing is completed, and step 300 is entered.

3. The method for picture text recognition based on layout optimization voting according to claim 2 is characterized in that: Step 300 includes, Step 301, read the pre-processed image Image, and read the Image size as h in height and w in width; Step 302, horizontally project the Image, and record the sum of the pixel values ​​of each row of the Image pixel matrix to obtain a horizontal projection vector Hor_proj; Step 303, read the horizontal projection vector Hor_proj, and record the position si where the value changes from 0 and the position ei where the value changes to 0 from top to bottom, and obtain the top and bottom positions of the row rowi = [si, ei], i = 1, 2, 3, ..., n, and all row positions row_all = [row1, row2, row3, ..., rown]; Step 304, traverse row_all, initialize the list row_width, calculate the absolute value of the difference between two elements in rowi, add it to the list row_width, and take the arithmetic mean width_mean of all elements in row_width; Step 305, traverse row_all and divide the image Image into Image_rowi=Image[rowi[0]:rowi[1],0:w], get the focused row image set Image_row_set=[Image_row1,Image_row2,…,Image_rown]; Step 306, traverse Image_row_set, perform vertical projection on Image_rowi, and record the sum of the pixel values ​​of each column in the image pixel matrix to obtain the vertical projection vector Ver_proji; Step 307, read the vertical projection vector Ver_proji corresponding to the row image Image_rowi in sequence, and record the position si where the value changes from 0 and the position ei where the value changes to 0 in sequence from left to right, and obtain the left column position and the right column position coli = [si, ei], and all column positions col_alli = [col1, col2, col3, ..., coln]; Step 308, read the column position list col_alli corresponding to the row image Image_rowi in turn, perform distance threshold-based clustering on the elements in the list col_alli, traverse col_alli, and if |colj[0]-colj-1[1]|≤t, no processing is performed; otherwise, colj and colj-1 are merged to obtain [colj-1[0], colj[1]], j=1, 2, 3, ..., n; the corresponding merged column position list col_merge_alli=[col_merge1, col_merge2, ..., col_mergem] is obtained by processing; Step 309, traverse all row images Image_rowi, read the merged column position list co_merge_alli corresponding to the size rowi_h and rowi_w of Image_rowi, and divide Image_rowi according to the column position recorded in col_merge_alli. rowi_colj=Image_rowi[0:rowi_h,col_mergej[0]:col_mergej[1]], j=1, 2, …, m, obtain the corresponding column image set Image_rowi_col_set={rowi_col1, rowi_col2, …, rowi_colm}; Step 310, initialize the sub-image width list sub_col_width, traverse all the column image sets Image_rowi_col_set corresponding to the row images Image_rowi, read the sub-image size width rowi_colj_w, add sub_col_width and the number of sub-images rowi_col_cnt; Step 311, sum all elements in the sub_col_width corresponding to Image_rowi to obtain w_sumi, initialize the size of the blank canvas Image_rowi_adjust to rowi_h*(w_sumi+int(1 / 3)*w_mean*(rowi_col_cnt-1)), integrate the sub-images in the corresponding column image set Image_rowi_col_set into Image_rowi_adjust in sequence, and the sub-image spacing is int(1 / 3)*w_mean; process all row images to obtain the adjusted row image set Image_row_adjust_set = {Image_row1_adjust, ..., Image_rowm_adjust} set; Step 312, traverse the adjusted row image set Image_row_adjust_set, use OpenCV to adjust the background size of a single row image Image_rowi_adjust, and fill the four sides of the row image with blanks using the corresponding row height rowi_h as the step size, and the n step size filling size is (rowi_h+2*n*rowi_h)*(rowi_w+2*rowi_w), and each row image is filled with 5 unit steps to obtain a set of adjusted image sets rowi_adjust_bk_set={rowi_h,rowi_2h,rowi_3h,rowi_4h,rowi_5h};Perform the same processing on all row images of all images;After image and text layout adjustment is completed, go to step 400.

4. The method for picture text recognition based on layout optimization voting according to claim 3 is characterized in that: The step 400 includes: Step 401, initialize the text row multi-layout recognition result list rowi_result, and the row image multi-layout image set rowi_adjust_bk_set = {rowi_h, rowi_2h, rowi_3h, rowi_4h, rowi_5h}; Step 402, traverse the multi-layout image set rowi_adjust_bk_set, input all images in the set into the text recognition engine for recognition, obtain recognition results, and add them to the recognition result list rowi_result, The final recognition result list rowi_result = [rowi_result_h, rowi_result_2h, rowi_result_3h, rowi_result_4h,rowi_result_5h]; Step 403, vote on the results in rowi_result, and the results with empty recognition results are not included in the vote. The recognition result with the highest frequency of occurrence is taken as the final recognition result of row rowi. Recognition and result voting are performed on all row images, and they are added to the result list Result in sequence, and then go to step 500.

5. A method for image text recognition based on layout optimization voting according to any one of claims 1 to 4, characterized in that: The text image file in step 100 includes jpg, png or tif.

6. The method for picture text recognition based on layout optimization voting according to claim 3 or 4, characterized in that: In step 308, the distance threshold is t=1*width_mean, where width_mean is the average line height of the image text.

7. A picture text recognition device based on layout optimization voting, characterized in that: It includes a memory, a control processor, and a computer program stored in the memory and executable on the control processor, wherein the control processor executes the program to implement the image text recognition method based on layout optimization voting as described in any one of claims 1 to 6.

8. A control system, characterized in that: It includes the image text recognition device based on layout optimization voting as described in claim 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the image text recognition method based on layout optimization voting as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Multi-classifier integration-based image character recognition method

    CN106127198A

  • Automatic character and picture segmentation system for recognizing image characters

    CN106611175A

  • Characteristic picture automatic generation system for image and character recognition

    CN106778759A

  • Image processing method, electronic circuit, visual impairment assistance device and medium

    WO2022237893A1