Character recognition method and system based on layout recognition

Abstract

A character recognition method and system for improving the accuracy of interpretation of text is provided. [Solution] The system includes a character-related information extraction unit including a deep learning model trained to extract character area information, character spacing information, line spacing scale information for each character, and character orientation information from an image containing text; a word unit segmentation recognition unit that acquires word segmentation information that divides characters contained in the image into words based on the character area information and character spacing information; a text line recognition unit that recognizes text lines in the image based on the character area information, line spacing scale information, and orientation information; a layout analysis unit that acquires text layout information based on the recognized text lines; and a character recognition unit that recognizes each character contained in the image and acquires text data in which the recognized characters are aligned based on the word segmentation information and layout information.

Description

Technical Field

[0001] The present invention relates to a character recognition method and system, and more particularly, to a method that enables accurate interpretation of text by recognizing the layout of text included in an image and providing a recognition result in which characters are recognized based on the recognized layout.

Background Art

[0002] Optical Character Recognition (OCR) is a technology that converts text included in a document image or the like into a text format that can be read by a computer, and is one of the core technologies that promote digital conversion in each industry. Such OCR technology is continuously evolving, and particularly, with the introduction of deep learning technology, the recognition rate and accuracy are greatly improving.

[0003] On the other hand, recently, while technologies such as language models and generative AI have emerged and developed, not only simply recognizing the characters themselves in an image, but also technologies for interpreting the content of text have become important. However, since conventional OCR technology is relatively vulnerable to recognizing the layout of text included in an image, even if each character is accurately recognized, it often cannot align the recognized characters to correspond to the layout of the text, which causes errors when interpreting the converted text.

Summary of the Invention

Problems to be Solved by the Invention

[0004] One problem to be solved by the present invention is to improve the interpretation accuracy of text by correctly aligning the characters recognized from an image in a computer system that utilizes character recognition technology such as OCR.

[0005] One problem that this invention aims to solve is to provide a method for more accurately recognizing the layout of text lines and other elements within an image. [Means for solving the problem]

[0006] To solve the aforementioned problems, a character recognition system according to one aspect of the present invention includes: a character-related information extraction unit including a deep learning model trained to extract character area information, blank area information between characters, line spacing scale information for each character, and orientation information for each character from an image containing text; a word unit segmentation recognition unit that acquires word segmentation information by segmenting the characters contained in the image into word units based on the character area information and blank area information between characters; a text line recognition unit that recognizes text lines in the image based on the character area information, line spacing scale information, and orientation information; a layout analysis unit that acquires layout information of the text contained in the image based on the recognized text lines; and a character recognition unit that recognizes each character contained in the image and acquires text data by aligning the recognized characters based on the word segmentation information and layout information.

[0007] In one embodiment, the character area information includes information about the area in the image where a character is inferred to be located, the blank area information between characters includes information inferred from the blank area existing between adjacent characters, the line spacing scale information includes information about the spacing between text lines determined by the position of each character, and the orientation information includes information about the angle of the text lines determined by the position of each character.

[0008] In one embodiment, the text line recognition unit defines element regions for each character to be determined based on the character region information, and recognizes text lines in the image based on whether the defined element regions are connected or overlapping.

[0009] In one embodiment, the text line recognition unit defines a first element region corresponding to the initial element region for each character to be determined based on the character region information, clusters the characters into a text line candidate set based on whether the defined first element regions are connected or overlapped, the center position of the first element region corresponds to the center position of the corresponding character, and the rotation angle of the first element region corresponds to the orientation information of the corresponding character.

[0010] In one embodiment, the text line recognition unit clusters characters into a set of text line candidates based on whether the second element regions, which are larger in size than the first element region, are connected or overlapped, and recognizes text lines for the characters contained in the image based on the clustering results.

[0011] In one embodiment, if the clustering result satisfies predetermined conditions, the text line recognition unit clusters the characters into a set of text line candidates based on the connection or overlap between the third element regions, which are formed by increasing the size of the second element region. If the clustering result does not satisfy predetermined conditions, the unit recognizes each of the clustered text line candidate sets as a single text line based on the first element region.

[0012] In one embodiment, the text line recognition unit defines a polynomial that minimizes the approximation error between the center point coordinates of each character included in the same set of candidate text lines and the polynomial itself, and determines whether the relationship between the approximation error for the defined polynomial and the average value of the line spacing scale information for each character satisfies the predetermined conditions.

[0013] In one embodiment, the layout analysis unit generates paragraph information that divides the text of the image into paragraphs based on the spacing between the recognized text lines, and generates line number information that divides the line numbers based on the y-axis intercept and center coordinates of each of the recognized text lines, and the layout information includes the paragraph information and the line number information.

[0014] A character recognition method according to one aspect of the present invention includes the steps of: extracting character-related information for a plurality of characters from an image containing text composed of a plurality of characters; obtaining word segmentation information by dividing the plurality of characters contained in the image into word units based on the extracted character-related information; recognizing text lines in the image based on the extracted character-related information; obtaining layout information of the text contained in the image based on the recognized text lines; and recognizing each of the plurality of characters contained in the image and obtaining text data by aligning the recognized characters based on the word segmentation information and the layout information. [Effects of the Invention]

[0015] Unlike conventional character recognition technologies, the character recognition method according to the present invention can improve the accuracy of text interpretation by recognizing the layout of text within an image and providing text data in which characters are aligned based on the recognized layout.

[0016] Furthermore, the method utilizes a character-related information extraction model trained to extract line spacing scale information and orientation information for each character contained in the image, thereby enabling accurate differentiation and recognition of text lines within a document.

[0017] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawing]

[0018] [Figure 1] This is a drawing showing a schematic configuration of a character recognition system according to an embodiment of the present invention. [Figure 2] This is a drawing showing an example configuration of the layout recognition unit shown in FIG. 1. [Figure 3] This is a drawing showing an example of information extracted from an input image by the character-related information extraction unit shown in FIG. 2 using a character-related information extraction model. [Figure 4] This is an exemplary diagram visualizing information extracted through the character-related information extraction model shown in FIG. 3. [Figure 5] This is an exemplary diagram visualizing information extracted through the character-related information extraction model shown in FIG. 3. [Figure 6A] This is an exemplary diagram visualizing the recognition result of the word unit classification recognition unit shown in FIG. 2. [Figure 6B] This is an exemplary diagram visualizing the recognition result of the word unit classification recognition unit shown in FIG. 2. [Figure 7] This is a drawing for a specific explanation of the text line recognition unit shown in FIG. 2. [Figure 8] This is a drawing for a specific explanation of the text line recognition unit shown in FIG. 2. [Figure 9] This is an exemplary diagram visualizing the text layout analysis result by the layout analysis unit shown in FIG. 2. [Figure 10] This is a flowchart for explaining a character recognition method according to an embodiment of the present invention. [Figure 11] This is a schematic hardware configuration block diagram of an arithmetic device constituting a character recognition system according to an embodiment of the present invention.

Mode for Carrying Out the Invention

[0019] FIG. 1 is a drawing showing a schematic configuration of a character recognition system according to an embodiment of the present invention.

[0020] Referring to Figure 1, the character recognition system 1 uses optical character recognition (Optical This system recognizes text using character recognition (OCR) technology and outputs text data converted into text format. Such a character recognition system 1 is configured to include at least one arithmetic unit. For example, each of the at least one arithmetic unit includes a hardware base device including a processor, memory, communication unit, input unit, and / or output unit. In this case, the configuration (module) included in the character recognition system 1 is embodied in hardware, software, or a combination thereof, and is embodied by being integrated or divided within the at least one arithmetic unit. Furthermore, the configuration (module) included in the character recognition system 1 is embodied in a computer-readable recording medium that stores at least one program in one form that includes command words for performing layout recognition and / or character recognition, as described later.

[0021] According to embodiments of the present invention, the character recognition system 1 includes a layout recognition unit 10 that recognizes the layout of text (e.g., a document) contained in an input image, and a character recognition unit 20 that recognizes each character contained in the image and provides text data in which the recognized characters are aligned based on the layout. The layout recognition unit 10 will be described in more detail later with reference to Figures 2 to 9. The character recognition unit 20 recognizes each character contained in the image based on various known OCR techniques, and depending on the embodiment, it may also recognize each character using a deep learning-based model (e.g., the Tesseract engine).

[0022] In other words, the character recognition system 1 according to an embodiment of the present invention recognizes the layout of text lines, paragraph numbers, line numbers, etc., of the text contained in the image, and provides text data in which the characters are aligned according to the recognized layout, thereby minimizing the problem of characters being misaligned and the content of the text not being correctly interpreted.

[0023] On the other hand, the result data output by such character recognition system 1 further includes data related to the layout of the text within the image (paragraph numbers, line numbers, etc.) in addition to the text data.

[0024] Figure 2 is a diagram showing an example configuration of the layout recognition unit shown in Figure 1. Figure 3 is a diagram showing an example of information extracted from an input image by the character-related information extraction unit shown in Figure 2 using the character-related information extraction model. Figures 4 and 5 are illustrative diagrams visualizing the information extracted through the character-related information extraction model shown in Figure 3. Figure 6 is an illustrative diagram visualizing the recognition results of the word-unit segmentation recognition unit shown in Figure 2. Figures 7 and 8 are diagrams for a detailed explanation of the text line recognition unit shown in Figure 2. Figure 9 is an illustrative diagram visualizing the text layout analysis results by the layout analysis unit shown in Figure 2.

[0025] Referring to Figure 2, the layout recognition unit 10 includes a character-related information extraction unit 110, a word-unit segmentation recognition unit 130, a text line recognition unit 150, and a layout analysis unit 170.

[0026] The character-related information extraction unit 110 extracts various information about the position, size, and / or orientation of characters from the input image. Referring to the embodiment in Figure 3, the character-related information extraction unit 110 includes a deep learning-based character-related information extraction model 112. For example, the character-related information extraction model 112 is realized by modification and fine-tuning based on various publicly known object sensing / segmentation models, but is not necessarily limited to those.

[0027] According to this embodiment, the character-related information extraction model 112 extracts character area information, blank area information between characters, and line spacing scale (interline) from the input image. It is implemented to infer and extract scale information and orientation information.

[0028] Character area information refers to information indicating the area within the input image where a character is inferred to be located, while inter-character whitespace information refers to information inferred from the whitespace between adjacent characters. For example, inter-character whitespace information indicates the whitespace between a specific character and the characters adjacent to its left and right, but is not necessarily limited to that.

[0029] For example, character area information indicates the probability (score) that each pixel corresponds to a character area, and the space area information between characters indicates the probability (score) that each pixel corresponds to a space between characters. In this case, the character area information and the space area information between characters are visualized in the form of heatmap images 410 and 420, respectively, as shown in Figure 4.

[0030] Referring again to Figure 2, the line spacing scale information indicates information about the spacing between text lines determined by the position of each character, and the orientation information indicates information about the angle (or orientation) of the text lines determined by the position of each character. For example, the line spacing scale information corresponds to the line spacing when there is a text line above or below each line, and to a predetermined multiple of the height of the text present in the line (e.g., twice) when there is no text around it. The line spacing is determined based on the top edge of the text in each line.

[0031] Referring to the example diagram in Figure 5, a visualized image 510 is shown, which visualizes the line spacing scale information and orientation information for each character contained in image 500. Visualized image 510 shows the line spacing scale information and orientation information via indicators 511 corresponding to each character. For example, the length of indicator 511 indicates the line spacing scale, and the angle (or direction) of indicator 511 corresponds perpendicularly to the angle (or direction) of the text line determined by the position of the character.

[0032] Referring again to Figure 2, the word-unit segmentation recognition unit 130 segmentes and recognizes the characters contained in the image on a word-by-word basis based on the character area information and the blank area information between characters output from the character-related information extraction unit 110, and provides the word segmentation information based on the recognition result to the character recognition unit 20.

[0033] Referring to the example diagram in Figure 6, the image 600 shown in Figure 6A is the image input to the character recognition system 1, and the image 610 shown in Figure 6B is a visualized image of the result of recognizing the characters contained in the input image 600 by dividing them into word units. For example, the visualized image 610 shows the word-unit division results in the form of text boxes, and the same serial number is displayed for each character belonging to the same text box. Depending on the embodiment, the visualized image 610 may also display an indicator showing line spacing scale information.

[0034] Referring again to Figure 2, the text line recognition unit 150 recognizes text lines in the input image based on the information extracted from the character-related information extraction unit 110. For example, the text line recognition unit 150 recognizes text lines based on character area information, line spacing scale information, and orientation information extracted from the character-related information extraction unit 110.

[0035] A specific example of the text line recognition method of the text line recognition unit 150 will be explained below with reference to Figures 7 and 8.

[0036] Referring to Figure 7, the text line recognition unit 150 defines an initial element area (first element area 702) for each character to be determined based on the character area information. For example, the first element area 702 is defined as a rectangle whose horizontal length is longer than its vertical length, and has the same size for each character. This is because text lines are generally formed horizontally, but in some embodiments, if an image in which text lines are formed vertically is input, the first element area 702 may be defined as a rectangle whose vertical length is longer than its horizontal length.

[0037] The center position of the first element region 702 corresponds to the center position of the corresponding character, and the rotation angle of the first element region 702 corresponds to the orientation information (angle or direction) of the corresponding character.

[0038] The text line recognition unit 150 can perform initial clustering of characters based on an initial element region (first element region 702) defined for each character. For example, the text line recognition unit 150 can perform initial clustering by grouping characters whose first element regions 702 are linked (or superimposed) into a candidate set. The first image 720 shown in Figure 7 is a visualization of the initial clustering result based on the first element regions 702, and it can be seen that the element regions of characters belonging to the same candidate set are represented by the same hue.

[0039] After the initial clustering, the text line recognition unit 150 repeats the clustering process while increasing the size of the element region. This is done until the clustering result no longer satisfies the predetermined conditions or until the predetermined conditions are met, and the text line recognition result is output based on the last clustering result.

[0040] Referring to the example diagram in Figure 7, the first image 720 visualizes the text line recognition results based on the initial clustering results. It can be seen that during the initial clustering, the size of the element area was small, and even though the characters belonged to the same text line, they were recognized as belonging to different text lines.

[0041] The text line recognition unit 150 can perform a clustering process based on a second element region 704 which is an enlarged version of the first element region 702. For example, the second element region 704 corresponds to a region whose horizontal size is increased from the first element region 702, and in some embodiments, it may correspond to a region in which both the horizontal and vertical sizes are increased while maintaining the horizontal-to-vertical ratio. In this case, the center position and rotation angle of the second element region 704 are the same as those of the first element region 702.

[0042] The text line recognition unit 150 clusters characters that are estimated to belong to the same text line into a candidate set based on a second element region 704 defined for each character. The second image 740 shown in Figure 7 is a visualization of the clustering results based on the second element region 704, and it can be seen that the text lines are recognized more accurately than in the first image 720.

[0043] On the other hand, if the size of the element region increases continuously, there is a risk that two or more text lines may be mistakenly identified as the same text line. For example, if the clustering process is performed based on the third element region 706, which has increased in size from the second element region 704, there is a risk that some text lines, even though they are different text lines, may be mistakenly identified as the same text line, as in the third image 760.

[0044] Therefore, if the clustering result no longer satisfies the predetermined conditions (or if the predetermined conditions are met for the first time), the text line recognition unit 150 terminates the clustering process and recognizes the text line based on the final clustering result that satisfies the predetermined conditions.

[0045] On the other hand, the shape of the element region described above can be defined in various ways. Referring to Figure 8, unlike the embodiment in Figure 7, the element region is defined as a circle. For example, the text line recognition unit 150 defines the initial element region for each character to correspond to the character region information extracted by the character-related information extraction unit 110. Furthermore, when the size of the element region increases, the text line recognition unit 150 can increase the size in the direction corresponding to the orientation information that has already been extracted, and in some embodiments, the size may be increased while maintaining the eccentricity of the element region (circle).

[0046] To describe the embodiment in Figure 8 in more detail, the text line recognition unit 150 defines an initial element area (first element area) for each character contained in the input image 800, and clusters characters that are estimated to belong to the same text line into a candidate set based on how the first element areas overlap (or connect). Referring to the visualized image 810 of the initial clustering result, it can be seen that characters belonging to the same text line have been recognized as belonging to different text lines. The text line recognition unit 150 performs clustering while increasing the size of the element areas and determines whether the clustering result satisfies the conditions given by the following formula 1.

[0047]

number

[0048] JPEG0007871436000002.jpg80152

[0049] The text line recognition unit 150 can repeat clustering while increasing the size of the element region until the clustering result does not satisfy the predetermined conditions. In Figure 8, if the clustering result 820 based on the second element region satisfies the predetermined conditions, and the clustering result 830 based on the third element region with an increased size of the second element region does not satisfy the predetermined conditions, the text line recognition unit 150 can output the text line recognition result using the last clustering result that satisfies the predetermined conditions (the clustering result based on the second element region).

[0050] Let's explain Figure 2 again.

[0051] The layout analysis unit 170 generates layout information by analyzing the layout of the text in the image based on the text line recognition results. For example, the layout information may include, but is not limited to, paragraph information that divides the text in the image into paragraphs and line number information based on the recognized text lines.

[0052] Referring together with the visualization image 900 in Figure 9, the layout analysis unit 170 generates paragraph information that divides paragraphs based on the spacing between recognized text lines. In some embodiments, the paragraph information further includes a paragraph number set for each paragraph based on the center coordinates of each divided paragraph. The layout analysis unit 170 can also generate line number information based on the y-axis intercept and center coordinates of each recognized text line.

[0053] The character recognition unit 20 recognizes each character contained in the input image. Based on the word segmentation information provided by the word segmentation recognition unit 130 and the layout information provided by the layout analysis unit 170, the character recognition unit 20 can acquire and output text data in which each of the recognized characters is aligned.

[0054] Figure 10 is a flowchart illustrating a character recognition method according to an embodiment of the present invention.

[0055] Referring to Figure 10, the character recognition system 1 extracts character-related information from the input image (S100), and based on the extracted character-related information, obtains word segmentation information that divides the characters contained in the image into word units (S110).

[0056] The character recognition system 1 clusters the characters into units of the same text line based on the extracted character-related information (S120), and obtains layout information of the text contained in the image based on the clustering results (S130).

[0057] The character recognition system 1 recognizes each character contained in the image using character recognition techniques such as OCR (S140), and acquires and outputs text data by arranging the recognized characters based on word segmentation information and layout information (S150).

[0058] Figure 11 is a schematic hardware block diagram of the arithmetic unit constituting the character recognition system according to an embodiment of the present invention.

[0059] The hardware configuration of the arithmetic unit 1100 shown in Figure 11 corresponds to the hardware configuration of each of the at least one arithmetic units that make up the character recognition system described above.

[0060] Referring to Figure 11, the arithmetic unit 1100 comprises a communication unit 1110, an input unit 1120, an output unit 1130, a control unit 1140, and a memory 1150. The control configuration shown in Figure 11 is an example for the sake of explanation, and the arithmetic unit 1100 may have more or fewer components than those shown in Figure 11.

[0061] The communication unit 1110 includes one or more communication modules that enable communication with other terminals, servers, etc., by connecting the computing unit 1100 to a network. For example, the communication modules include mobile communication modules such as LTE and 5G, wireless communication modules such as Wi-Fi, and / or other various wired or wireless communication modules.

[0062] The input unit 1120 is configured to acquire information such as user input, images, and audio, and includes a variety of input means such as various mechanical / electronic input means, cameras, and microphones. The output unit 1130 is for providing information to the user by generating outputs related to sight, hearing, or touch, and includes a display, speaker, vibration module, etc.

[0063] The control unit 1140 controls the overall operation of the arithmetic unit 1100. The control unit 1140 processes signals, data, information, etc. that are input or output through the aforementioned components, or provides predetermined information and functions through various applications and algorithms stored in the memory 1150. For example, the control unit 1140 controls the overall process of the character recognition method disclosed herein.

[0064] Such a control unit 1140 includes at least one processor and / or at least one programmable circuit. For example, the control unit 1140 includes a CPU (central processing unit), AP (application processor), MCU, GPU (graphics processing unit), NPU (neural Processing unit, integrated circuit, ASIC (application-specific integrated circuit), FPGA (field-programmable) This is implemented using hardware such as gate arrays.

[0065] Memory 1150 stores programs and data necessary for the operation of the arithmetic unit 1100. Memory 1150 also stores data generated or acquired through the control unit 1140. Memory 1150 is a ROM (Read Only Memory), RAM (Random Access Memory), Flash Memory, SSD (Solid State Drive), HDD (Hard Drive) It consists of a recording medium such as a disk drive or a combination of recording media.

[0066] The embodiments of the present invention described above can be embodied as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all recording devices on which data to be read by a computer system is stored. Examples of computer-readable media include HDDs, SSDs, and SDDs (Silicon These include disk drives, ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0067] The examples in this disclosure relate to two national research and development projects. be The information for the first national research and development project is as follows: Project identification number 1711197986, project number RS-2023-00255968, project name "Training of Human Resources for Artificial Intelligence Fusion Innovation (Ministry of Science and ICT)", project name "Training of Human Resources for Artificial Intelligence Fusion Innovation (Ajou University)". be Information on another national research and development project is available under project identification number 1711193301, project number IITP-2024-2020-0-01461, project name "Support Project for University ICT Research Centers", and project name "Development of Intelligent Medical Image Diagnostic Solutions". be . [Explanation of symbols]

[0068] 10: Layout recognition unit 20: Character recognition section 110: Character-related information extraction unit 130: Word-level segmentation recognition unit 150: Text line recognition unit 170: Layout Analysis Department

Claims

1. In a character recognition system including at least one computing unit, A character-related information extraction unit includes a deep learning model trained to extract character area information, spacing information between characters, line spacing scale information for each character, and orientation information for each character from an image containing text. A word unit segmentation recognition unit obtains word segmentation information by segmenting the characters contained in the image into word units, based on the character area information and the blank space information between characters. A text line recognition unit recognizes text lines within the image based on the character area information, line spacing scale information, and orientation information. A layout analysis unit obtains layout information of the text contained in the image based on the recognized text lines, A character recognition system including a character recognition unit that recognizes each character contained in the aforementioned image and obtains text data in which the recognized characters are aligned based on the word segmentation information and layout information.

2. The character region information includes information about the region in the image where it is inferred that a character is located. The information regarding the blank space between characters includes information obtained by inferring the blank space that exists between adjacent characters. The aforementioned line spacing scale information includes information about the spacing between text lines determined at the position of each character, The character recognition system according to claim 1, wherein the orientation information includes information regarding the angle of the text line determined at the position of each character.

3. The text line recognition unit, Based on the aforementioned character area information, define an element area for each character to be determined, The character recognition system according to claim 1, which recognizes text lines in an image based on whether defined element regions are connected or superimposed.

4. The text line recognition unit, Based on the aforementioned character area information, a first element area corresponding to the initial element area for each character to be determined is defined. Based on how the defined first element regions are connected or overlapped, characters are clustered into a set of text line candidates. The character recognition system according to claim 3, wherein the center position of the first element region corresponds to the center position of the corresponding character, and the rotation angle of the first element region corresponds to the orientation information of the corresponding character.

5. The text line recognition unit, Based on how the second element regions, whose size has been increased, are connected or overlapped, the characters are clustered into a set of text line candidates. The character recognition system according to claim 4, which recognizes text lines for characters contained in the image based on the clustering results.

6. The text line recognition unit, If the clustering result satisfies predetermined conditions, Based on how the third element regions, whose size has been increased, are connected or overlapped, the characters are clustered into a set of text line candidates. If the clustering result does not satisfy the predetermined conditions, The character recognition system according to claim 5, which recognizes each clustered set of text line candidates as a single text line based on the first element region.

7. The text line recognition unit, We define a polynomial that minimizes the approximation error between the center point coordinates of each character included in the same set of candidate text lines, The character recognition system according to claim 6, which determines whether the relationship between the approximation error for a defined polynomial and the average value of the line spacing scale information for each of the characters satisfies the predetermined conditions.

8. The aforementioned layout analysis unit, Based on the spacing between the recognized text lines, paragraph information is generated that divides the text of the image into paragraphs. Based on the y-axis intercept and center coordinates of each of the recognized text lines, line number information is generated, separating the line numbers. The character recognition system according to claim 1, wherein the layout information includes the paragraph information and the line number information.

9. A step of extracting character-related information for a plurality of characters from an image containing text composed of a plurality of characters, using a deep learning model trained to extract information about the whitespace between characters, the line spacing scale information for each character, and the orientation information for each character, Based on the extracted character-related information, the step of obtaining word segmentation information by dividing the multiple characters contained in the image into word units, The steps include: recognizing text lines in the image based on extracted character-related information; The steps include obtaining layout information of the text contained in the image based on the recognized text lines, A character recognition method comprising the steps of: recognizing each of the multiple characters contained in the aforementioned image; and obtaining text data in which the recognized characters are aligned based on the word segmentation information and the layout information.

10. The aforementioned character-related information is, Character region information including information about the region in the image where it is inferred that a character is located, Intercharacter whitespace information, which includes information inferred from the whitespace between adjacent characters, Line spacing scale information including information about the spacing between text lines determined at the position of each of the aforementioned multiple characters, The character recognition method according to claim 9, further comprising orientation information including information regarding the angle of a text line determined at the position of each of the aforementioned plurality of characters.

11. The step of recognizing the aforementioned text line is: The steps include defining an element area for each character determined based on the aforementioned character area information, A character recognition method according to claim 10, comprising the step of recognizing text lines in an image based on whether defined element regions are connected or superimposed.

12. The step of recognizing the aforementioned text line is: The steps include defining a first element region corresponding to the initial element region for each character determined based on the aforementioned character region information, The process includes the step of clustering characters into a set of text line candidates based on whether they are connected or overlapped between defined first element regions, The character recognition method according to claim 11, wherein the center position of the first element region corresponds to the center position of the corresponding character, and the rotation angle of the first element region corresponds to the orientation information of the corresponding character.

13. The step of recognizing the aforementioned text line is: The steps of clustering characters into a set of text line candidates based on whether the second element regions, which have increased size, are connected or overlapped, A character recognition method according to claim 12, comprising the step of recognizing text lines for characters contained in the image based on the clustering results.

14. The step of recognizing the aforementioned text line is: If the clustering result satisfies predetermined conditions, the characters are clustered into a set of text line candidates based on whether the third element regions, whose size has been increased, are connected or overlapped. The character recognition method according to claim 13, further comprising the step of recognizing each clustered set of candidate text lines as a single text line based on the first element region if the clustering result does not satisfy predetermined conditions.

15. Based on the clustering results, the step of recognizing text lines for characters contained in the image is: The steps include defining a polynomial that minimizes the approximation error between the center point coordinates of each character included in the same set of candidate text lines and the polynomial itself, The character recognition method according to claim 14, comprising the step of determining whether the relationship between the approximation error for a defined polynomial and the average value of the line spacing scale information for each of the characters satisfies the predetermined conditions.

16. The step of obtaining the aforementioned layout information is: The steps include generating paragraph information that divides the text of the image into paragraphs based on the spacing between the recognized text lines, The character recognition method according to claim 9, comprising the step of generating line number information that divides the line numbers based on the y-axis intercept and center coordinates of each of the recognized text lines.

Images