Image processing system, control method of image processing system, and program

Abstract

To provide a mechanism which allows for improvement of a character recognition rate.SOLUTION: An MFP 100 comprises: recognition means (character recognition unit 116) which recognizes a character included in an image; printing means (printing unit 113) which prints an image including the character recognized by the recognition means and an entry column in which a character can be hand-written, to obtain a printed matter (a re-learning list print 501); reading means (reading unit 115) which reads the character in the printed matter and a hand-written character written in the entry column; and learning control means (learning unit 119) which accepts input of a character read by the reading means as a first character and accepts input of a hand-written character written in the entry column as a second character to perform control to train a learning model with the first character and the second character in association with each other.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an image processing system, a control method for an image processing system, and a program. [Background technology]

[0002] Conventionally, there have been handwritten character recognition devices that recognize handwritten characters. Known examples of such handwritten character recognition devices include a device that executes character recognition processing to recognize handwritten characters using an OCR (Optical Character Reader) (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-97370 Summary of the Invention [Problem to be solved by the invention]

[0004] Some writers have idiosyncratic handwriting. In such cases, character recognition using machine learning, for example, may not be accurate, leading to a lower recognition rate.

[0005] The present invention has been made in view of the above-mentioned problems, and has an object to provide a mechanism that can improve the character recognition rate. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the image processing system of the present invention is characterized by comprising: a recognition means for recognizing characters contained in an image; a printing means for printing an image including the characters recognized by the recognition means and an entry field in which the characters can be written by hand to obtain the printed matter; a reading means for reading the characters in the printed matter and the handwritten characters written in the entry field; and a learning control means for inputting the characters read by the reading means as first characters into a learning model and inputting the handwritten characters written in the entry field as second characters, and controlling the learning to associate the first characters with the second characters. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the character recognition rate. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration when the image processing system according to the first embodiment is applied to an MFP (Multi-Function Peripheral). [Figure 2] 3A and 3B are diagrams showing a read image read by a reading unit and a recognition result obtained by recognizing characters included in the read image; [Figure 3] 10 is a flowchart showing a process for determining whether a character is stored in a low-reliability character storage unit. [Figure 4] FIG. 10 is a diagram showing a list of characters stored in a low-reliability character storage unit. [Figure 5] FIG. 10 is a diagram illustrating an example of a printed list of characters stored in a low-reliability character storage unit. [Figure 6] FIG. 10 is a diagram showing a modified example of a printed list of characters stored in a low-reliability character storage unit. [Figure 7] FIG. 10 is a block diagram showing a re-learning process. [Figure 8] FIG. 10 is a diagram illustrating an example of a screen displayed when correcting the learning results of a trained model. [Figure 9]FIG. 11 is a diagram showing an example of an operation image for a list of characters stored in a low-reliability character storage unit in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in each embodiment. For example, each component constituting the present invention can be replaced with any configuration that can perform the same function. Also, any component may be added. Furthermore, any two or more configurations (features) of each embodiment can be combined.

[0010] First Embodiment A first embodiment will be described below with reference to FIGS. 1 to 8. FIG. 1 is a block diagram showing an example of the configuration when an image processing system according to the first embodiment is applied to an MFP (Multi-Function Peripheral). As shown in FIG. 1, MFP 100 has a CPU 101, a ROM 102, a RAM 103, a storage 104, a communication unit I / F 105, a communication connector 106, an operation unit I / F 107, an operation unit 108, a UI control unit (display control means) 109, and a display unit 110. MFP 100 also has an image processing unit 111, a printer controller I / F 112, a printing unit (printing means) 113, a scanner controller I / F 114, a reading unit (reading means) 115, and a GPU 140. These pieces of hardware included in MFP 100 are connected to each other so as to be able to communicate with each other via a system bus 130. CPU 101 controls the overall operation of MFP 100. The CPU 101 reads control programs stored in the ROM 102 or the storage 104 to perform various control operations, such as scanning control and printing control. The ROM 102 stores control programs executable by the CPU 101 and a boot program executed at startup. The RAM 103 is the main memory of the CPU 101. The RAM 103 is also used as a work area and a temporary storage area for loading the various control programs stored in the ROM 102 and the storage 104. The storage 104 stores print data, image data, various programs, and various setting information. The programs include, for example, programs for causing a computer to execute the various means of the image processing system. The GPU 140 is a processor for neural network calculations and is used in character recognition processing, which will be described later. Note that this character recognition processing may be performed by the GPU 140 alone, by the CPU 101 alone, or by both the GPU 140 and the CPU 101. The MFP 100 may also use, for example, a TPU instead of the GPU 140.

[0011] An operation unit 108 is connected to the operation unit I / F 107. The operation unit 108 is not particularly limited and may be, for example, a keyboard or a mouse. A reading unit 115 is connected to the scanner controller I / F 114. The reading unit 115 reads an image on a sheet and generates image data thereof. This image data is, for example, printed on a sheet by the printing unit 113 or transmitted to an external device (not shown) communicatively connected to the MFP 100. The image data is also used as input for OCR (optical character recognition). A printing unit 113 is connected to the printer controller I / F 112. Image data to be printed by the printing unit 113 is transmitted to the printing unit 113 via the printer controller I / F 112. This allows the printing unit 113 to receive control commands and image data to be printed. The printing unit 113 can then print an image based on the image data to be printed on a sheet. The printing method of the printing unit 113 is not particularly limited, and may be, for example, an electrophotographic method or an inkjet method. The image processing unit 111 rotates, compresses, and converts the resolution of image data output to the printing unit 113 via the printer controller I / F 112. A communication connector 106 is connected to the communication unit I / F 105. The communication connector 106 is connected to an external device (not shown) such as a server via a network (not shown). A display unit 110 is connected to the UI control unit 109. The UI control unit 109 controls the display of images on the display unit 110. The display unit 110 is not particularly limited, and may be, for example, an LCD.

[0012] FIG. 2 shows a scanned image scanned by the scanning unit 115 and the recognition results of characters included in the scanned image. Scanned image 201 in FIG. 2 is an image obtained by scanning by the scanning unit 115. Scanned image 201 includes the handwritten character string "Test." Table 202 in FIG. 2 shows the recognition results of the characters included in scanned image 201 recognized by the character recognition unit 116 (see FIG. 1). Table 202 includes, as headings, recognized character 203, reliability 204, and image 205. Image 205 includes handwritten characters "T," "e," "s," and "t." Note that image 205 may include the entire image data and coordinate data of the scanned character image instead of the handwritten character image. Recognized character 203 includes a "T" in a predetermined font as the result of recognizing the handwritten "T." Similarly, recognized characters 203 include an "e" in a predetermined font as a result of recognizing a handwritten "e," an "s" in a predetermined font as a result of recognizing a handwritten "s," and a "t" in a predetermined font as a result of recognizing a handwritten "t." Reliability 204 includes the reliability (degree of confidence) when a handwritten character is recognized as a character in a predetermined font. For example, the reliability when a handwritten character "T" is recognized as a character "T" in a predetermined font is "0.9893915." Note that in this embodiment, characters also include symbols.

[0013] However, depending on the writer, handwritten characters may have idiosyncratic writing. In this case, character recognition processing by the character recognition unit 116 may result in a decrease in the accuracy of character recognition. For example, depending on the degree of idiosyncratic writing, a handwritten character "t" may be recognized as the character "+" in a specific font, or may be completely unrecognizable. Therefore, the MFP 100 is configured to suppress such a decrease in character recognition rate. The configuration and operation of this unit are described below. As shown in FIG. 1, the storage 104 stores, as software, the character recognition unit (recognition means) 116, the reliability calculation unit (computation means) 117, the low-reliability character storage unit (storage means) 118, the learning unit 119, and the layout unit 120. The functions of these pieces of software are executed by one of the CPU 101 and the GPU 140.

[0014] The character recognition unit 116 recognizes, for example, handwritten characters included in an image using OCR, and further recognizes the recognized characters using a trained model (trained model) trained by a known machine learning method (recognition process). Deep learning is preferable as a machine learning algorithm for the trained model, but is not limited to this and may be, for example, a support vector machine, a logistic regression, a decision tree, etc. The reliability calculation unit 117 calculates (calculates) the reliability of the characters recognized by the character recognition unit 116 as the likelihood of the recognition result by the character recognition unit 116. For example, when the character recognition unit 116 recognizes characters a predetermined number of times, this reliability is calculated as the proportion of the number of times that the characters recognized by the character recognition unit 116 are correctly recognized within the predetermined number of times.

[0015] In this embodiment, CPU 101 also functions as a determination unit that determines whether a character recognized by character recognition unit 116 is to be stored in low-reliability character storage unit 118, based on the reliability calculated by reliability calculation unit 117. For example, if the reliability is less than a threshold, CPU 101 determines the character to be stored, and if the reliability is equal to or greater than the threshold, CPU 101 determines the character not to be stored. For example, in FIG. 4 (described later), if the reliability is less than the threshold "0.7," the character having the reliability is determined to be stored, and if the reliability is equal to or greater than the threshold "0.7," the character having the reliability is determined to be not to be stored. Note that MFP 100 can also change the threshold. In this case, the threshold is pre-stored in storage 104 and can be changed as appropriate by operating operation unit 108. In this way, operation unit 108 also functions as a change unit that changes the threshold. In addition to characters determined to be stored by CPU 101, low-reliability character storage unit 118 stores the reliability of the characters and images of the characters. The low-reliability character storage unit 118 only needs to be able to store at least the characters that are determined to be the storage target.

[0016] The learning unit 119 is a program (learning control means) that repeatedly performs relearning processing on the trained model (character recognition unit 116) using a known machine learning method. In this embodiment, the trained model is relearned using a VAE (Varional Autoencoder). Learning using the trained model will be described later. The layout unit 120 arranges, on printing paper, characters and the like included in the relearning list print 501 shown in FIG. 5 and the relearning list print 601 shown in FIG. 6 that are output from the printing unit 113.

[0017] 3 is a flowchart showing the process of determining whether or not a character is stored in the low-reliability character storage unit. A program based on the flowchart shown in FIG. 3 is stored in the low-reliability character storage unit 118. This program is then read into the RAM 103 and executed by the CPU 101. As shown in FIG. 3, in step S301, the CPU 101 determines whether or not the reading unit 115 has read an image (document) containing characters. If it is determined in step S301 that the reading unit 115 has read the image, the process proceeds to step S302. On the other hand, if it is determined in step S301 that the reading unit 115 has not read the image, the process remains in standby at step S301.

[0018] In step S302, the CPU 101 stores the scanned image generated by the image reading by the reading unit 115 in the storage 104.

[0019] In step S303, CPU 101 recognizes characters included in the scanned image stored in storage 104 in step S302 using character recognition unit 116 and learning unit 119. Then, CPU 101 causes reliability calculation unit 117 to calculate the reliability of the characters, which are the recognition results.

[0020] In step S304, CPU 101 determines whether the reliability (degree of reliability) calculated in step S303 is smaller than a threshold value. If the result of the determination in step S304 is that it is smaller than the threshold value, the process proceeds to step S305. On the other hand, if the result of the determination in step S304 is that it is not smaller than the threshold value, that is, that it is equal to or greater than the threshold value, the process ends.

[0021] In step S305, CPU 101 stores the characters determined to be smaller than the threshold value in step S304 in low-reliability character storage unit 118, and the process ends.

[0022] FIG. 4 is a diagram showing a list of characters stored in the low-reliability character storage unit. The low-reliability character storage unit 118 stores a low-reliability character list 401 shown in FIG. 4. The low-reliability character list 401 includes an item (header) 402, an item 403, an item 404, and an item 405. The item 402 is a list number, and in FIG. 4, it includes "1" to "7." The item 403 is a character (character to be stored) stored in step S305, and in FIG. 4, it includes "a" linked to the list number "1." The item 404 is a reliability, and in FIG. 4, it includes "0.6982143" linked to the list number "1." The item 405 is an image of a character stored in step S305, and in FIG. 4, it includes an image of "a" linked to the list number "1." The low-reliability character list 401 is created for each user who uses the MFP 100. This allows handwritten character learning, which will be described later, to be performed for each user.

[0023] FIG. 5 is a diagram illustrating an example of a printed list of characters stored in the low-confidence character storage unit. The relearning list print 501 shown in FIG. 5 is a printout obtained by printing an image of the list of characters stored in the low-confidence character storage unit 118 using the printing unit 113 (printing process). This relearning list print 501 is used for relearning using a trained model. The relearning list print 501 includes a relearning character area 502 that is read by the reading unit 115. The relearning character area 502 includes multiple pairs of paired recognized character areas 503 and user entry areas 504, and also includes a learning mode setting mark 505. The recognized character area 503 contains characters (characters to be memorized) listed in item 403 of the low-confidence character list 401 shown in FIG. 4. A user entry area 504 is arranged adjacent to the recognized character area 503 below the recognized character area 503. The user entry area 504 is an entry field where characters in the recognized character area 503 can be written by hand. For example, if the character "a" is entered in the recognized character area 503, the character "a" can be handwritten in the user entry area 504 below the recognized character area 503. In this way, the re-learning character area 502 lists a plurality of recognized character areas 503 and the user entry areas 504 that are paired with each recognized character area 503, i.e., that correspond to each recognized character area 503. The layout of this list is performed by the layout unit 120. The layout image obtained by the layout unit 120 is then converted into a print image by the image processing unit 111 and printed by the printing unit 113. In this way, the re-learning list print 501 is obtained.

[0024] The timing at which the relearning list print 501 is printed is not particularly limited, and can be, for example, the timing when the list number included in the item 402 reaches a predetermined value (threshold value) or more. Alternatively, the timing can be the timing after a certain time has elapsed since the previous printing of the relearning list print 501, or any timing determined by the user. A message recommending printing the relearning list print 501 may be displayed on the display unit 110. While the relearning list print 501 is a single sheet of printed material in the configuration shown in FIG. 5, the present invention is not limited to this, and the relearning list print 501 may be a multi-sheet printed material, for example.

[0025] Furthermore, the relearning list print 501 with handwritten characters entered in the recognized character area 503 is read by the reading unit 115 and stored in the storage 104 as a read image (scanned image). Thereafter, relearning is performed by the learning unit 119 using a trained model in response to a user operation or in a re-learning mode. The learning mode setting mark 505 is a mark indicating that the relearning list print 501 is a mark. When the character recognition unit 116 recognizes the learning mode setting mark 505, relearning may be performed using a trained model.

[0026] FIG. 6 is a diagram showing a modified example of a printed list of characters stored in the low-reliability character storage unit. Similar to the relearning list print 501 shown in FIG. 5, the relearning list print 601 shown in FIG. 6 is a printed matter obtained by printing an image of the list of characters stored in the low-reliability character storage unit 118 by the printing unit 113. The relearning list print 601 includes a relearning character area 607 that is read by the reading unit 115 in the same way as the relearning character area 502, and a learning mode setting mark 608 that has a function similar to the learning mode setting mark 505. The relearning character area 607 includes items 602, 603, 604, 605, and 606. Item 602 includes list numbers "1" to "7" similar to item 402 in the low-reliability character list 401 shown in FIG. 4. Item 603 is a character (character to be memorized) similar to item 403, and includes characters such as "あ" associated with list number "1." Item 604 has the same reliability as item 404, and includes "0.6982143" linked to list number "1". Item 605 is an image of a character similar to item 405, and includes an image of the character "a" linked to list number "1". Item 606 is an entry field where the character in item 603 can be written by hand. For example, if item 603 contains "a", then "a" can be written by hand in item 606.

[0027] FIG. 7 is a block diagram showing the relearning process. In the relearning process, either the relearning list print 501 (see FIG. 5) or the relearning list print 601 (see FIG. 6) can be used. Here, the relearning list print 501 is used as an example. First, the reading unit 115 reads the characters in the recognized character area 503 in the relearning list print 501 and the handwritten characters entered in the user entry area 504 (reading process). As described above, the learning unit 119 is a program that causes the learning model to perform the relearning process. The characters in the recognized character area 503, i.e., label data 702 that serves as the correct character data, are input as the first character to parameter adjustment 703 of the learning model. In addition to the first character input to parameter adjustment 703, the handwritten characters entered in the user entry area 504, i.e., image data 701 of the handwritten characters, are input as the second character.

[0028] The learning model to which the first and second characters are input learns to associate the first and second characters, i.e., to recognize the second character as the first character (learning control process). This learning is repeated. As a result, the learned model outputs the re-trained model, relearned character recognition unit 704, as an optimal inference result. Then, relearned character recognition unit 704 can be used for character recognition processing (OCR) instead of character recognition unit 116. As a result, for example, if the handwritten character "a" is written in a habitual style on a document to be subjected to character recognition processing, if the handwritten character "a" is similar to the recognized character "a" already recognized by relearned character recognition unit 704, the handwritten character "a" can be recognized as the character "a". In this way, MFP 100 can accurately recognize handwritten characters even if the handwritten character is habitual, thereby improving the character recognition rate.

[0029] FIG. 8 is a diagram showing an example of a screen displayed when correcting the learning results of a trained model. A home screen 800 shown in FIG. 8(a) is a diagram showing an example of a screen displayed on the display unit 110 of the MFP 100. The home screen 800 includes a machine learning icon 801. By operating, i.e., pressing, the machine learning icon 801, the operation instruction is transmitted from the operation unit 108 to the CPU 101 via the operation unit I / F 107. As a result, the CPU 101 displays a machine learning OCR setting screen 802 shown in FIG. 8(b) on the display unit 110. The machine learning OCR setting screen 802 includes a menu button 803 as a selection menu. By operating the menu button 803, the CPU 101 displays a character list display screen 804 shown in FIG. 8(c) on the display unit 110. The character list display screen 804 is a screen showing a list of characters with low reliability, and includes items 805, 806, 807, 808, and 809. Item 805 is a list number, and in FIG. 8(c) includes "1" to "6." Item 806 is a recognized character (character to be memorized), and includes "a" linked to list number "1," etc. Item 807 is reliability, and includes "0.6982143" linked to list number "1." Item 808 is a character image, and includes an image of "a" linked to list number "1." Item 809 is an entry field where the recognized character can be corrected and entered. If the user determines that the recognized character in item 806 of list number "1" is different from the image of the character in item 808, the user operates recognized character correction button 810 for list number "1." This operation causes CPU 101 to display soft keyboard 812 shown in FIG. 8(d) on display unit 110. The user then operates soft keyboard 812 to input the correct character, i.e., the same character as the recognized character in item 806, into item 809 of list number "1." This character is input as label data 702 (see FIG. 7) to parameter adjustment 703 for the learning model. This results in re-learning character recognition unit 704, which further improves the character recognition rate. In this way, MFP 100 can forcibly correct the learning results of the learning model. In this embodiment, character list display screen 804 functions as correction means for forcibly correcting the learning results of the learning model.

[0030] The character list display screen 804 also includes a print button 811. By operating the print button 811, a printout of a list including items 805, 806, 807, 808, and 809, such as the re-learning list printout 501 shown in FIG.

[0031] Second Embodiment The second embodiment will be described below with reference to FIG. 9. The description will focus on differences from the previous embodiment, and similar details will not be repeated. FIG. 9 is a diagram showing an example of an operation image for a list of characters stored in the low-reliability character storage unit in the second embodiment. The relearning list image 901 shown in FIG. 9 is an image (operation image) of a list of characters stored in the low-reliability character storage unit 118, and is displayed on the display unit 110. In this embodiment, the display unit 110 has a touch panel function. This allows operations to be performed on the relearning list image 901. Similar to the relearning list print 501, this relearning list image 901 is used for relearning a trained model. The relearning list image 901 includes a relearning character area 902 that is read by the CPU 101 as a reading unit. The relearning character area 902 includes multiple pairs of paired recognized character areas 903 and user entry areas 904, and further includes a learning mode setting mark 905. Similar to recognized character area 503, recognized character area 903 contains characters (characters to be stored) listed in item 403 of low-reliability character list 401 shown in Fig. 4. Below recognized character area 903, user entry area 904 is arranged adjacent to it. User entry area 904 is an entry field where the characters in recognized character area 903 can be written by hand using a touch pen or the like.

[0032] Furthermore, the relearning list image 901 with handwritten characters entered in the recognized character area 903 is read by the CPU 101 and saved as a read image in the storage 104. Relearning is performed using a trained model. The learning mode setting mark 905 is a mark indicating that the relearning list image 901 is an image for relearning. By checking the learning mode setting mark 905, the user can understand that the relearning list image 901 is an image for relearning.

[0033] Although preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and various modifications and variations are possible within the scope of the gist thereof. The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions. Furthermore, although the device to which the image processing system is applied is an MFP in the above-described embodiments, this is not limited to this and may be any device that has at least a scanning function. Furthermore, the trained model is not limited to being stored in the MFP 100 and may be stored in an external device, such as a server connected to the MFP 100 so as to be able to communicate with it.

[0034] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) A recognition means for recognizing characters included in an image; a printing means for printing an image including the characters recognized by the recognition means and an entry field in which the characters can be written by hand, to obtain a printed matter; reading means for reading the characters in the printed matter and the handwritten characters entered in the entry fields; An image processing system characterized by comprising: a learning control means that inputs the character read by the reading means as the first character and the handwritten character entered in the entry field as the second character into a learning model, and controls the learning of associating the first character with the second character. (Configuration 2) The image processing system according to configuration 1, wherein the learning model is trained to recognize the second character as the first character. (Configuration 3) The image processing system according to configuration 1 or 2, wherein the learning model performs learning using a VAE (Variational Autoencoder). (Configuration 4) The image processing system according to any one of configurations 1 to 3, wherein the learning control means repeatedly causes the learning model to learn the first character and the second character by associating them with each other. (Configuration 5) A determination means is provided to determine whether or not the character is to be stored depending on the reliability of the recognition result by the recognition means, 5. The image processing system according to any one of configurations 1 to 4, wherein the printing means includes the characters determined by the determining means to be stored in the printed matter. (Configuration 6) The image processing system according to Configuration 5, wherein the judgment means determines that the character is to be stored if the reliability is smaller than a threshold value, and determines that the character is not to be stored if the reliability is equal to or greater than the threshold value. (Configuration 7) The image processing system according to configuration 6, further comprising a change unit for changing the threshold value. (Configuration 8) The image processing system according to any one of configurations 5 to 7, further comprising a calculation means for calculating the reliability. (Configuration 9) The image processing system according to any one of configurations 5 to 7, further comprising storage means capable of storing at least the characters that are determined by the determination means to be the characters to be stored. (Configuration 10) An image processing system described in any one of configurations 1 to 9, characterized in that the printing means acquires the printed matter as a printed matter in which a plurality of the characters and the entry fields paired with each of the characters are listed. (Configuration 11) The image processing system according to any one of configurations 1 to 10, further comprising a correction means for forcibly correcting the learning result of the learning model. (Configuration 12) A recognition means for recognizing characters included in an image; a display control means for controlling the display of an operation image including the characters recognized by the recognition means and an entry field in which the characters can be written by hand; a reading means for reading the characters in the operation image and the handwritten characters entered in the entry field; and a learning control means for inputting the character read by the reading means as a first character and inputting the handwritten character entered in the entry field as a second character, and controlling the association and learning of the first character and the second character. (Method 1) A method for controlling an image processing system, comprising: a recognition step of recognizing characters contained in the image; a printing step of printing an image including the characters recognized by the recognition step and an entry field in which the characters can be written by hand to obtain a printed matter; a reading step of reading the characters in the printed matter and handwritten characters entered in the entry field; A control method for an image processing system, comprising: a learning control process for inputting the character read in the reading process as a first character and the handwritten character entered in the entry field as a second character into a learning model, and controlling the learning of the first character and the second character in association with each other. (Program 1) A program for causing a computer to execute each means of the image processing system according to any one of configurations 1 to 12. [Explanation of symbols]

[0035] 100 MFP 109 UI control section 113 Printing Department 115 Reading unit 116 Character recognition section 117 Reliability Calculation Unit 118 Low-reliability character memory 119 Learning Department 120 Layout Section 501 Re-learning list print

Claims

1. recognition means for recognizing characters contained in an image; a printing means for printing an image including the characters recognized by the recognition means and an entry field in which the characters can be written by hand, to obtain a printed matter; reading means for reading the characters in the printed matter and the handwritten characters entered in the entry fields; An image processing system characterized by comprising a learning control means that inputs the character read by the reading means as a first character and the handwritten character entered in the entry field as a second character into a learning model, and controls the learning of associating the first character with the second character.

2. 2. The image processing system according to claim 1, wherein the learning model is trained to recognize the second character as the first character.

3. 2. The image processing system according to claim 1, wherein the learning model is learned using a VAE (Variational Autoencoder).

4. 2. The image processing system according to claim 1, wherein the learning control means repeatedly causes the learning model to learn the first character and the second character by associating them with each other.

5. a determination means for determining whether or not the character is to be stored depending on the reliability of the recognition result by the recognition means; 2. The image processing system according to claim 1, wherein the printing means includes the characters determined by the determining means to be stored in the printed matter.

6. The image processing system according to claim 5, wherein the determination means determines that the character is to be stored if the reliability is smaller than a threshold value, and determines that the character is not to be stored if the reliability is equal to or greater than the threshold value.

7. 7. The image processing system according to claim 6, further comprising a change unit for changing the threshold value.

8. 6. An image processing system according to claim 5, further comprising a calculation means for calculating the reliability.

9. 6. The image processing system according to claim 5, further comprising storage means for storing at least the characters that are determined by the determination means to be the characters to be stored.

10. 2. The image processing system according to claim 1, wherein the printing means acquires the printed matter as a printed matter in which a plurality of the characters and the entry fields paired with each of the characters are listed.

11. 2. The image processing system according to claim 1, further comprising a correction means for forcibly correcting the learning results of the learning model.

12. recognition means for recognizing characters contained in an image; a display control means for controlling the display of an operation image including the characters recognized by the recognition means and an entry field in which the characters can be written by hand; a reading means for reading the characters in the operation image and the handwritten characters entered in the entry field; and a learning control means for inputting the character read by the reading means as a first character and inputting the handwritten character entered in the entry field as a second character, and controlling the association and learning of the first character and the second character.

13. 1. A method for controlling an image processing system, comprising: a recognition step of recognizing characters contained in the image; a printing step of printing an image including the characters recognized by the recognition step and an entry field in which the characters can be written by hand to obtain a printed matter; a reading step of reading the characters in the printed matter and handwritten characters entered in the entry field; A control method for an image processing system, characterized by comprising a learning control process for inputting the character read in the reading process as a first character and inputting the handwritten character entered in the entry field as a second character into a learning model, and controlling the learning of the first character and the second character in association with each other.

14. 2. A program for causing a computer to execute each means of the image processing system according to claim 1.

Images