Character recognition system and character recognition method

Abstract

A character recognition system 1 comprises a storage unit 3 for storing a layout and an extraction target position in a form to be read, an imaging unit 2 for capturing an image of the form, and a control unit 5 for recognizing a plurality of constituent elements of the form from the image. The control unit 5 extracts a character string at the extraction target position on the basis of a comparison between the arrangement of the plurality of recognized constituent elements and the layout stored in the storage unit 3.

Description

Character recognition system and character recognition method

[0001] The present invention relates to a character recognition system and a character recognition method.

[0002] Optical character recognition (OCR) technology, which captures images of printed or handwritten characters and reads text information, has been widely used. In particular, in product management and delivery management at logistics centers, forms with product-related text printed in a predetermined format and layout are attached to products, and reading the text information from the form image using OCR processing can contribute to the automation of subsequent logistics processes.

[0003] Patent Document 1 discloses a conventional technique for capturing an image of a printed label, performing OCR processing, and converting desired written items into text information for extraction. More specifically, this conventional technique employs a method for extracting ruled lines separating characters from a captured image of a form, and recognizing the names of the written items to identify the extraction target position and read the desired character string.

[0004] Japanese Patent Application Laid-Open No. 2017-129926

[0005] However, with the above-mentioned conventional technology, even if the desired character string itself is readable, the reading position may not be identified if the ruled lines or item names used to identify the extraction target position are unrecognizable due to dirt or if they are not within the angle of view. Furthermore, for example, in the case of a form without item names or ruled lines, the extraction target position cannot be identified, which may lead to an error in recognizing the desired character string.

[0006] The present invention has been made in view of the above circumstances, and its purpose is to provide a character recognition system and a character recognition method that can extract a character string at a desired location even when a specific positional reference such as a ruled line or a character string that has been specified in advance such as an item name cannot be recognized.

[0007] In order to achieve the above-mentioned object, the character recognition system of the present invention comprises a memory unit that stores the layout and extraction target position of a form to be read, an imaging unit that captures an image of the form, and a control unit that recognizes multiple components of the form from the image, and the control unit extracts a character string at the extraction target position based on a comparison of the arrangement of the multiple recognized components with the layout stored in the memory unit.

[0008] In addition, in order to achieve the above-mentioned object, the character recognition method of the present invention includes a storage step of storing the layout and extraction target position of a form to be read, an imaging step of capturing an image of the form, a recognition step of recognizing multiple components of the form from the image, and a character string extraction step of extracting a character string at the extraction target position based on a comparison of the arrangement of the recognized multiple components with the stored layout.

[0009] According to the present invention, it is possible to provide a character recognition system and a character recognition method that can extract a character string from a desired location even when a specific positional reference such as a ruled line or a character string that is pre-specified such as an item name cannot be recognized.

[0010] FIG. 1 is a block diagram showing the main configuration of a character recognition system according to the present invention. FIG. 2 is a diagram showing an example of a first form. FIG. 3 is a diagram showing an example of a second form. FIG. 4 is a flowchart showing the execution procedure of a form registration mode. FIG. 5 is a flowchart showing the execution procedure of a character recognition method. FIG. 6 is an example of a touch panel screen when imaging a form. FIG. 7 is an example of a touch panel screen when recognizing components. FIG. 8 is a conceptual diagram schematically showing the state of internal processing for comparing the arrangement of components. FIG. 9 is an example of a confirmation screen showing the reading results.

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the content described below, and can be implemented with any modifications within the scope of the gist of the present invention. Furthermore, all drawings used to explain the embodiments are schematic representations of components, and may be partially emphasized, enlarged, reduced, or omitted to facilitate understanding, and may not accurately represent the scale, shape, etc. of the components.

[0012] 1 is a block diagram showing the main components of a character recognition system 1 according to the present invention. In this embodiment, the character recognition system 1 is a smartphone capable of executing a character recognition application, which will be described later, and includes an imaging unit 2, a storage unit 3, a touch panel 4, and a control unit 5. The character recognition system 1 is used, for example, when a user reads a desired character string from a form attached to a package.

[0013] The character recognition system 1 includes other components such as a communication module that is included in a typical smartphone. The character recognition system 1 is not limited to a smartphone, and may be other portable devices such as a tablet terminal, smart glasses, a head-mounted display, or a handheld terminal.

[0014] The imaging unit 2 is a camera that captures an image of a form to be read while a character recognition application is running. The imaging unit 2 can also be used to obtain preliminary information about the form to be read prior to the execution of the character recognition application, as will be described in detail later.

[0015] The storage unit 3 is a storage for storing the character recognition application, advance information on the form to be read, character recognition results, and the like.

[0016] The touch panel 4 is an input / output interface that combines a display as the display unit 4a and a touch sensor as the input unit 4b, and can display images captured by the imaging unit 2 in real time and accept touch operations from the user.

[0017] The control unit 5 is a CPU (Central Processing Unit) that receives information from each component of the character recognition system 1 and controls the operation of each component, and controls the reading of desired character strings from a form by executing a character recognition application.

[0018] Here, examples of multiple forms in this embodiment will be described. Fig. 2 is a diagram showing an example of a first form F1. Fig. 3 is a diagram showing an example of a second form F2. Note that the forms handled by the character recognition system 1 are not limited to these, and may be one type, or three or more types.

[0019] As shown in Figure 2, the first form F1 contains multiple item names i1 to i5 representing product numbers, model numbers, etc., and values ​​v1 to v5 are written in positions corresponding to each item name. Also, as shown by i5, a unit of quantity may be treated as an item name. The first form F1 does not have any lines separating the item names or their values, and the layout is determined by the arrangement of these components that make up the form.

[0020] 3, the second form F2 lists a plurality of item names i1 to i6 representing the product number, manufacturing date, etc., and lists values ​​v1 to v6 in positions corresponding to each item name. Although the item names and their values ​​are separated by ruled lines, the second form F2 is generally laid out in three blocks: product details, weight, and date, with two parallel ruled lines positioned closely between the blocks.

[0021] Next, a form registration mode will be described in which information about a form is stored in advance in the storage unit 3 so that the character recognition system 1 can perform character recognition. Fig. 4 is a flowchart showing the execution procedure of the form registration mode. A form can be pre-registered by several methods, but here we will describe a method of registering a form using the character recognition system 1 itself. The user can start the form registration mode by operating the character recognition system 1 via the input unit 4b.

[0022] In the form registration mode, when a user holds the imaging unit 2 over a printed form of the type to be read, the control unit 5 acquires an image of the form via the imaging unit 2 (step S1). Here, the imaging unit 2 is positioned in front of the form, and the form is imaged without tilting so that the height and width directions of the form match the vertical and horizontal directions of the image, respectively.

[0023] The control unit 5 also recognizes the plurality of components of the form from the captured image, that is, in the case of the first form F1, the plurality of item names i1 to i5 and values ​​v1 to v6 (step S2).

[0024] At this time, if the target form is the first form F1, the control unit 5 performs character recognition on all character strings in the first form F1 by OCR processing. Furthermore, if the target form is the second form F2, the control unit 5 may recognize the ruled lines in addition to all character strings in the second form F2. Here, a known algorithm can be used to recognize the components, and for example, the recognition function of MLkit, which is released as an SDK by Google, can be adopted.

[0025] Once the components are recognized, the control unit 5 calculates the coordinates of each component (step S3). More specifically, the control unit 5 calculates the coordinates of the four corners of the character string "No" for the item name i1 of the first form F1. Similarly, the control unit 5 calculates the coordinates of the components for each of the item names i1 to i5 and values ​​v1 to v6.

[0026] Once the coordinates of the components have been calculated, the control unit 5 prompts the user to select which of the component positions of the form is the extraction target position (step S4). The user specifies the position of the character string to be extracted from the multiple components recognized in step S3 via the input unit 4b as the extraction target position. Here, it is assumed that values ​​v1 and v3 are specified as the desired character strings.

[0027] As a result, the control unit 5 stores the coordinates of all the components recognized from the form as the layout of the form in the storage unit 3 together with the target extraction target position (step S5, storage step).

[0028] Then, if multiple forms are expected to be read, such as the first form F1 and the second form F2, the control unit 5 registers the layout and extraction target position for each form (No in step S6) and registers all forms (Yes in step S6).

[0029] Note that, here, an example is given of a method for registering the layout and extraction target position from an image of an actual printed form when that image is available, but if the original data of the form before printing exists, the layout and extraction target position can also be registered from that original data.

[0030] In addition, if a large number of form images can be acquired for each type of form, a classification model for determining the type of form may be created separately using a known machine learning algorithm and stored in the memory unit 3.

[0031] Next, a description will be given of a character recognition method executed by the character recognition system 1 in the operation stage. Fig. 5 is a flowchart showing the execution procedure of the character recognition method. A user starts a character recognition application of the character recognition system 1 to obtain a desired character string from, for example, a form attached to a product.

[0032] When the character recognition application is started, the control unit 5 starts the imaging unit 2 so that an image of the form to be read can be acquired. At this time, the control unit 5 acquires an image of the form via the imaging unit 2 when the user holds the imaging unit 2 over the form to be read (step S10, imaging step), and causes the image to be displayed on the touch panel 4 in real time.

[0033] 6 shows an example of the screen of the touch panel 4 when an image of a form is captured. Here, it is assumed that the first form F1 is captured, and the first form F1 is captured at an angle relative to the touch panel 4.

[0034] Furthermore, if a form classification model is stored in the storage unit 3, the control unit 5 determines the type of form in the captured image using the classification model (step S11). The determined form is displayed as master data M, for example, on a part of the touch panel 4, as shown in Fig. 6. In addition, a selection button B is displayed to allow the user to confirm whether the determination result is correct.

[0035] Here, the master data M can be displayed as image data in which the item names i1 to i5 remain in the form and the values ​​v1 to v5 have been deleted. The master data M displayed here may be the image captured in form registration mode as is, or, if original data of the form before printing exists, the original data may be displayed as is. Furthermore, the layout and design of the master data M and selection buttons B on the touch panel 4 are not limited to this and can be changed as appropriate. Furthermore, confirmation of the judgment result is not required, and the display of the master data M and selection buttons B may be omitted.

[0036] The control unit 5 also recognizes the plurality of components of the form, i.e., the plurality of item names i1 to i5 and values ​​v1 to v6, and their coordinates, from the captured image (step S12, recognition step). The component recognition here may involve converting the item names and values ​​into text data by OCR processing, as well as recognizing ruled lines and figures included in the form.

[0037] However, by recognizing only the characters of the form from the image as components, the control unit 5 eliminates the need for computationally intensive image processing in subsequent processes, thereby improving processing speed. Furthermore, by recognizing the components as characters, recognition accuracy can be ensured and errors in determining whether the components are the same or different can be reduced when comparing the components.

[0038] 7 shows an example of the screen of the touch panel 4 during element recognition. Element recognition can employ a recognition function based on a known algorithm, similar to the form registration mode described above. Here, the recognition results are displayed by encircling the recognized item names i2 to i5 with dashed lines and the values ​​v1 to v6 with solid lines. The lines encircling the item names and values ​​may be displayed in different colors, such as red or yellow. In the example of FIG. 7, the first form F1 is captured at an angle relative to the touch panel 4, and therefore the item name i1, which is outside the angle of view, is not recognized as an element.

[0039] The recognition results of the components are displayed on the touch panel 4 in real time for images continuously acquired from the imaging unit 2. Therefore, the user can adjust the character recognition system 1 as a portable medium by moving it relative to the form to be read so that at least the desired character string fits within the angle of view.

[0040] Also, when the components are recognized, the control unit 5 compares the arrangement of the multiple components recognized in step S11 with the layout of each form previously registered in the memory unit 3 in form registration mode (step S13).

[0041] 8 is a conceptual diagram showing the internal processing for comparing the arrangement of components. More specifically, the control unit 5 searches for corresponding points between the captured image IMG, which was captured while the first form F1 was tilted, and the master data M, and calculates a projection transformation matrix that converts the coordinates of the master data M to the coordinates of the form on the captured image IMG by, for example, associating common item names (step S14). In FIG. 8, the desired extraction target position in the master data M is indicated by a dashed line.

[0042] If a classification model has not been created in advance in form registration mode, the form cannot be determined in step S11. Therefore, for each type of registered form, the components are recognized (step S12) and their arrangement is compared (step S13), and instead of step S11, the form with the closest arrangement of the components is determined to be the form of the currently captured image.

[0043] Furthermore, the control unit 5 identifies, based on the projective transformation matrix, the coordinates of a position in the form on the captured image IMG that corresponds to the extraction target position in the master data M (step S15). The control unit 5 also extracts, from the components recognized in step S12, the element at the coordinates identified in step S15 as a desired character string (step S16, character string extraction step).

[0044] Then, the control unit 5 causes the touch panel 4 to display, side by side, the partial image data obtained by cutting out the portion of the captured image IMG at the extraction target position and the text data of the character string extracted in step S16 (step S17).

[0045] 9 is an example of a confirmation screen showing the reading result. In this embodiment, the desired character string consists of two items, value v1 and value v3, so the confirmation screen displays a first panel P1 and a second panel P2 corresponding to these items.

[0046] More specifically, the first panel P1 displays partial image data im obtained by cutting out a portion of the captured image IMG at the extraction target position corresponding to the value v1, alongside text data tx of the character string recognized at this extraction target position. Similarly, the second panel P2 displays partial image data im obtained by cutting out a portion of the captured image IMG at the extraction target position corresponding to the value v3, alongside text data tx of the character string recognized at this extraction target position. Even if the form is captured tilted in the captured image IMG, the tilt of the partial image data im is corrected based on the projective transformation matrix before it is displayed.

[0047] Additionally, a selection button B is displayed on the touch panel 4, allowing the user to confirm whether the desired character string has been correctly recognized from the image. If the recognition result is incorrect, character recognition can be performed again in steps S10 to S17, or the character string can be manually corrected via the touch panel 4. Furthermore, when scanning multiple forms consecutively, character recognition can be repeated in steps S10 to S17.

[0048] When it is confirmed that the desired character string has been extracted, the control unit 5 compiles the extracted character string as data, for example, a CSV file, and stores it in the memory unit 3 or another storage location (step S18), and then terminates the series of character recognition applications.

[0049] It is preferable that the control unit 5 separately transmits and stores the partial image data im at the position of the value v1 extracted as the desired character string from the captured image IMG to the storage unit 3 or another storage location. In this case, even if the character recognition is not performed correctly in step 12, correct recognition may be possible later by sending the data to, for example, an external OCR for handwritten characters.

[0050] As described above, the character recognition system 1 according to the present invention recognizes multiple components of a form from the captured image IMG of the form acquired via the imaging unit 2, and identifies the extraction target position in the captured image IMG by comparing the arrangement of these components with the form layout previously stored in the storage unit 3. Therefore, even if the character recognition system 1 cannot recognize part of an item name, such as a ruled line, it can identify the writing position of a desired character string based on the arrangement of the components in the entire form. Therefore, the character recognition system 1 can extract a character string from a desired location even if it cannot recognize specific positional references, such as ruled lines, or pre-specified character strings, such as item names.

[0051] <Embodiments of the Present Invention> A character recognition system according to a first embodiment of the present invention comprises a memory unit that stores the layout and extraction target position of a form to be read, an imaging unit that captures an image of the form, and a control unit that recognizes multiple components of the form from the image, and the control unit extracts a character string at the extraction target position based on a comparison of the arrangement of the multiple recognized components with the layout stored in the memory unit.

[0052] A character recognition system according to a first embodiment of the present invention recognizes multiple components of a form from a captured image of the form acquired via an imaging unit, and identifies the extraction target position in the captured image by comparing the arrangement of these components with a form layout previously stored in a storage unit. Therefore, even if the character recognition system cannot recognize some of the ruled lines or field names, it can identify the writing position based on the arrangement of the components in the entire form. Therefore, the character recognition system can extract character strings from desired locations even if it cannot recognize specific positional criteria such as ruled lines or pre-specified character strings such as field names.

[0053] A character recognition system according to a second embodiment of the present invention is the first embodiment of the present invention described above, wherein the memory unit stores the layout and the extraction target position for each of a plurality of types of form, and the control unit determines the type of form from the image captured by the imaging unit and extracts the character string.

[0054] According to the character recognition system of the second embodiment of the present invention, even if there are multiple types of forms to be read, by storing the layouts and extraction target positions of each in advance, it is possible to appropriately extract the desired character string from the extraction target positions that differ for each form.

[0055] A character recognition system according to a third embodiment of the present invention is the first or second embodiment of the present invention described above, and includes a display unit that displays the character recognition results, and the display unit displays partial image data obtained by cutting out the portion of the image corresponding to the extraction target position and the text data of the extracted character string side by side.

[0056] According to the character recognition system of the third embodiment of the present invention, the captured image of the extraction target position and the read result can be compared closely, so that the user can easily confirm whether the desired character string has been correctly recognized from the image.

[0057] A character recognition system according to a fourth embodiment of the present invention is the system according to the first or second embodiment of the present invention, and stores partial image data obtained by cutting out a portion of the image at the extraction target position.

[0058] According to the character recognition system of the fourth embodiment of the present invention, even if character recognition is not performed correctly when recognizing a component, correct recognition may be possible after the fact by sending partial image data to, for example, an external OCR for handwritten characters, so that, for example, multiple forms can be read continuously without having to check the recognition results each time.

[0059] A character recognition system according to a fifth embodiment of the present invention is the first or second embodiment of the present invention described above, wherein the control unit calculates a projection transformation matrix between the layout stored in the memory unit and the arrangement of the recognized plurality of components, and identifies the position of the extraction target in the image based on the projection transformation matrix.

[0060] According to the character recognition system of the fifth embodiment of the present invention, even if the form is not imaged from the front or is imaged at an angle, the acquired image of the form can be corrected to the arrangement of the form registered in advance.

[0061] A character recognition system according to a sixth embodiment of the present invention is the first or second embodiment of the present invention, wherein the control unit recognizes only the characters of the form from the image as the plurality of components.

[0062] According to the character recognition system of the sixth embodiment of the present invention, when recognizing elements from an image of a form, only characters are recognized as elements, rather than lines or marks, thereby eliminating the need for computationally intensive image processing in subsequent processing, thereby improving processing speed. Furthermore, by recognizing elements as characters, errors in determining whether elements are the same or different can be reduced when comparing elements.

[0063] A character recognition system according to a seventh embodiment of the present invention is the first or second embodiment of the present invention described above, and has a form registration mode in which the control unit calculates the coordinates of a plurality of the components recognized from the image to form the layout, and stores the position of a specified element among the components in the memory unit as the extraction target position.

[0064] According to the character recognition system of the seventh embodiment of the present invention, even if the original data of the form to be read is not available, if an image of the completed and printed form is available, the form can be registered without using any other device.

[0065] A character recognition method according to an eighth embodiment of the present invention includes a storage step of storing the layout and extraction target position of a form to be read, an imaging step of capturing an image of the form, a recognition step of recognizing a plurality of components of the form from the image, and a character string extraction step of extracting a character string at the extraction target position based on a comparison of the arrangement of the recognized plurality of components with the stored layout.

[0066] A character recognition method according to an eighth embodiment of the present invention recognizes multiple components of a form from a captured image of the form acquired in an imaging step, and identifies the extraction target position in the captured image by comparing the arrangement of these components with a pre-stored form layout. Therefore, even if some of the ruled lines or field names cannot be recognized, the character recognition method can identify the writing position based on the arrangement of the components in the entire form. Therefore, the character recognition method can extract a character string from a desired location even if specific positional criteria such as ruled lines or pre-specified character strings such as field names cannot be recognized.

[0067] A character recognition method according to a ninth embodiment of the present invention is the same as the eighth embodiment of the present invention, in which the storage step stores the layout and the extraction target position for each of a plurality of form types, and after determining the type of form from the image captured in the imaging step, the character string extraction step is executed.

[0068] According to the character recognition method of the ninth embodiment of the present invention, even if there are multiple types of forms to be read, by storing the layouts and extraction target positions of each in advance, it is possible to appropriately extract the desired character string from the extraction target positions that differ for each form.

[0069] A character recognition method according to a tenth embodiment of the present invention is the same as that according to the eighth or ninth embodiment of the present invention, in which partial image data cut out from the image at the extraction target position and text data of the extracted character string are displayed side by side.

[0070] The character recognition method according to the tenth embodiment of the present invention allows for close comparison of the captured image of the extraction target position with the read result, allowing the user to easily confirm whether the desired character string has been correctly recognized from the image.

[0071] A character recognition method according to an eleventh embodiment of the present invention is the same as that of the eighth or ninth embodiment of the present invention, and stores partial image data obtained by cutting out a portion of the image at the extraction target position.

[0072] According to the character recognition method of the eleventh embodiment of the present invention, even if character recognition cannot be performed correctly when recognizing a component, correct recognition may be possible after the fact by sending partial image data to, for example, an external OCR for handwritten characters, so that, for example, multiple forms can be read continuously without having to check the recognition results each time.

[0073] A character recognition method according to a twelfth embodiment of the present invention is the same as the eighth or ninth embodiment of the present invention, and in the character string extraction step, a projective transformation matrix between the stored layout and the arrangement of the recognized plurality of components is calculated, and the position of the extraction target in the image is identified based on the projective transformation matrix.

[0074] According to the character recognition method of the twelfth embodiment of the present invention, even if the form is not imaged from the front or is imaged at an angle, the acquired image of the form can be corrected to the arrangement of the form registered in advance.

[0075] A character recognition method according to a thirteenth embodiment of the present invention is the eighth or ninth embodiment of the present invention, wherein in the recognition step, only the characters of the form are recognized from the image as the plurality of components.

[0076] According to the character recognition method of the thirteenth embodiment of the present invention, when recognizing elements from an image of a form, only characters are recognized as elements, rather than lines or marks, thereby eliminating the need for computationally intensive image processing in subsequent processing, thereby improving processing speed. Furthermore, by recognizing elements as characters, errors in determining whether elements are the same or different can be reduced when comparing elements.

[0077] A character recognition method according to a fourteenth embodiment of the present invention is the eighth or ninth embodiment of the present invention described above, in which, in the storage step, coordinates of a plurality of the components recognized from the image are calculated to form the layout, and the position of a specified element among the components is stored as the extraction target position.

[0078] According to the character recognition method of the fourteenth embodiment of the present invention, even if the original data of the form to be read is not available, the form can be registered if an image of the completed and printed form is available.

[0079] 1 Character recognition system 2 Imaging unit 3 Memory unit 4 Touch panel 4a Display unit 4b Input unit 5 Control unit F1 First form F2 Second form M Master data B Selection button P1 First panel P2 Second panel im Partial image data tx Text data IMG Captured image i1 to i6 Item name v1 to v6 Value

Claims

1. A character recognition system comprising: a memory unit that stores the layout and extraction target position of a form to be read; an imaging unit that captures an image of the form; and a control unit that recognizes multiple components of the form from the image, wherein the control unit extracts a character string at the extraction target position based on a comparison of the arrangement of the multiple recognized components with the layout stored in the memory unit.

2. The character recognition system of claim 1, wherein the memory unit stores the layout and the extraction target position for each of a plurality of types of form, and the control unit determines the type of form from the image captured by the imaging unit and extracts the character string.

3. A character recognition system as described in claim 1 or 2, comprising a display unit that displays the character recognition results, wherein the display unit displays partial image data obtained by cutting out the portion of the image corresponding to the extraction target position and the text data of the extracted character string side by side.

4. A character recognition system according to claim 1 or 2, wherein partial image data obtained by cutting out the portion of the image where the extraction target is located is stored.

5. A character recognition system as described in claim 1 or 2, wherein the control unit calculates a projection transformation matrix between the layout stored in the memory unit and the arrangement of the recognized multiple components, and identifies the extraction target position in the image based on the projection transformation matrix.

6. A character recognition system according to claim 1 or 2, wherein the control unit recognizes only the characters of the form from the image as the plurality of components.

7. A character recognition system as described in claim 1 or 2, wherein the control unit has a form registration mode in which the coordinates of the plurality of components recognized from the image are calculated to form the layout, and the position of a specified element among the components is stored in the memory unit as the extraction target position.

8. A character recognition method comprising: a storage step of storing the layout and extraction target positions in a form to be read; an imaging step of capturing an image of the form; a recognition step of recognizing multiple components of the form from the image; and a character string extraction step of extracting a character string at the extraction target position based on a comparison of the arrangement of the recognized multiple components with the stored layout.

9. A character recognition method as described in claim 8, wherein in the storage step, the layout and the extraction target position are stored for each of a plurality of types of form, and after determining the type of form from the image captured in the imaging step, the character string extraction step is executed.

10. A character recognition method according to claim 8 or 9, wherein partial image data obtained by cutting out the portion of the image corresponding to the extraction target position and the extracted text data of the character string are displayed side by side.

11. A character recognition method according to claim 8 or 9, wherein partial image data obtained by cutting out the portion of the image at the extraction target position is saved.

12. A character recognition method as described in claim 8 or 9, wherein the character string extraction step calculates a projection transformation matrix between the stored layout and the arrangement of the recognized multiple components, and identifies the position of the extraction target in the image based on the projection transformation matrix.

13. A character recognition method according to claim 8 or 9, wherein in the recognition step, only the characters of the form are recognized from the image as the plurality of components.

14. A character recognition method as described in claim 8 or 9, wherein in the storage step, the coordinates of the plurality of components recognized from the image are calculated to form the layout, and the position of a specified element among the components is stored as the extraction target position.

Images