Verification system

Abstract

To provide means, when converting analog data consisting of handwritten characters into text data in data input, capable of converting the analog data with high recognition accuracy.SOLUTION: A verification system solves a problem by causing an AI function for classifying handwritten character image data on the basis of a combination of the handwritten character image data and text data corresponding to the handwritten character image data, to acquire AI recognition text data. The classification is performed on data that has undergone recognition accuracy improvement processing for improving the recognition accuracy of handwritten character image data, the processing for removing digital information existing around handwritten character image data, the digital data other than digital information corresponding to the handwritten characters.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] This invention relates to a technology used in a service called data entry or verify entry, which converts characters that have not been converted into text data into text data. In particular, this invention relates to a technology used in a service called data entry or verify entry, in which characters that have not been converted into text data are handwritten characters. [Background technology]

[0002] There are a large number of documents on the market that consist of characters that have not been converted into text data (analog data characters), and there is a service called "data entry" that undertakes the accurate conversion of analog data character information into text data.

[0003] Data entry is typically performed by a first worker acquiring primary text data and a second worker acquiring secondary text data. First, character image data, which is image information obtained from analog character data, and recognized text data, which is obtained from the character image data, are acquired from the document to be converted into text data. The first worker compares the character image data with the recognized text data, and any recognized text data determined to match the character image data is used as primary text data, while any recognized text data determined to not match the character image data is corrected to match the character image data and become primary text data. The second worker compares the character image data with the primary text data acquired by the first worker, and any primary text data determined to match the character image data is used as secondary text data, while any primary text data determined to not match the character image data is corrected to match the character image data and become secondary text data. In the data entry process, the secondary text data from the second worker is used as the text data obtained by converting analog character information into text data. In the data entry process, the judgments of the first and second workers are combined in this way to accurately convert character information consisting of analog data into text data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2013-97393 [Patent Document 2] Patent Publication No. 2008-152357 Summary of the Invention [Problem to be solved by the invention]

[0005] In data entry, the recognized text data used by the first-stage worker is obtained through OCR processing. When using OCR processing, if the target is printed text, the text recognition accuracy is extremely high, exceeding 90%. By being able to start the work of the first-stage worker with a recognition accuracy of over 90%, accurate conversion of character information consisting of analog data into text data is realized.

[0006] On the other hand, when the text data to be converted is handwritten, the accuracy of text recognition using OCR processing is very low. For handwritten items related to monetary amounts with several digits, when OCR processing was used to obtain recognized text data for the entire item, the character recognition rate was 27.0%. Therefore, when the data object is handwritten, the conversion to text data must be carried out carefully, which poses problems in terms of the time and effort required. The objective of this invention is to provide a means for converting analog data consisting of handwritten characters into text data with high recognition accuracy when converting the data into text data during data input.

[0007] The development of technology for accurately converting handwritten characters into text has not been realized in many fields, not just data entry, and development is desired. Patent Document 1 describes a handwritten character text conversion system capable of highly accurately reading handwritten characters written in a notebook. In this system, character image data captured with a digital camera mounted on a smartphone is transmitted to a center server via a communication network, where the character image data is converted into text data. The conversion to text data is performed by two or more operators reading the handwritten characters written in the handwritten space of the notebook and transmitting (inputting) the reading results from the operator terminal. Furthermore, the handwritten characters in the handwritten space are optically read and converted into text data using OCR, and the input by the operators is performed simultaneously.

[0008] Patent Document 2 describes a text recognition system that improves the efficiency of recognizing handwritten input information. In this system, image data of handwritten characters, such as the names of candidates written on a ballot paper, is converted into text data using OCR, and data that cannot be converted and is therefore unreadable is sent to a person in charge, who visually selects candidates from the handwritten characters. [Means for solving the problem]

[0009] (1) A verification system for acquiring text data corresponding to characters in analog data from characters in analog data, wherein the characters in the analog data are handwritten characters, the system includes: a primary text data acquisition means for acquiring primary text data recorded by a primary worker; a secondary text data acquisition means for acquiring secondary text data recorded by a secondary worker; and an AI-recognized text data acquisition means for classifying handwritten character image data, which is data obtained by digitizing the handwritten characters in the analog data, and acquiring AI-recognized text data, which is data recognized as text data corresponding to the handwritten character image data. The primary text data acquisition means compares the handwritten character image data with the AI-recognized text data, and acquires, as primary text data, any of the AI-recognized text data that is determined to match the handwritten character image data, and corrects any of the AI-recognized text data that is determined not to match the handwritten character image data to match the handwritten character image data. the handwritten character image data is acquired as primary text data in a state in which the handwritten character image data is identical to the handwritten character image data, the secondary text data acquisition means compares the handwritten character image data with the primary text data, and acquires, as secondary text data, any of the primary text data that is determined to match the handwritten character image data as is, and any of the primary text data that is determined not to match the handwritten character image data is corrected to match the handwritten character image data and acquired as secondary text data, the AI-recognized text data acquisition means acquires the AI-recognized text data by an AI function that classifies the handwritten character image data based on a combination of the handwritten character image data and text data corresponding to the handwritten character image data, the AI ​​function being a process of removing digital information that is present in the periphery of the handwritten character image data and is not digital information corresponding to handwritten characters, and the problem is solved by a verification system that classifies data that has been subjected to a recognition accuracy improvement process for improving the recognition accuracy of the handwritten character image data. (2) The problem is solved by the verification system described in (1), characterized in that in the AI-recognized text data acquisition means, the AI-recognized text data corresponding to one character is acquired by an AI function that classifies the handwritten character image data of one character based on a combination of handwritten character image data of one character that is one of the characters that make up the handwritten character and is obtained by dividing the handwritten character image data of the one character, and text data corresponding to the handwritten character image data of the one character. (3) The problem is solved by the verification system described in (1), characterized in that in the AI-recognized text data acquisition means, AI-recognized text data for a single-character number is acquired by an AI function that classifies handwritten character image data for a single-character number based on a combination of handwritten character image data for the single-character number and text data corresponding to the handwritten character image data for the single-character number. (4) The problem is solved by the verification system described in (1), characterized in that in the primary text data acquisition means, the handwritten character image data and the AI-recognized text data are compared by arranging the AI-recognized text data corresponding to each character based on the positional information of each character in the handwritten character image data. (5) In the recognition accuracy improvement process, for digital information other than digital information corresponding to handwritten characters that includes position information, the problem is solved by the verification system described in (1), which is characterized in that, based on the position information, pixels in locations where digital information other than digital information corresponding to handwritten characters exists are replaced with white pixels.

[0010] In cases where the characters in the analog data are handwritten, we considered training AI using a combination of image data of the handwritten characters and the correct text data corresponding to those characters as training data. The AI's functions make it possible to classify the various image data of handwritten characters into the text data corresponding to those handwritten characters. Through such classification, it is possible to obtain AI-recognized text data, which is data recognized as text data corresponding to the handwritten character image data.

[0011] Typically, information remains around handwritten character image data, such as lines that were written on the paper on which the handwritten characters are written before the characters are written, parts of other nearby handwritten characters, and dirt. This information is unnecessary when converting the target handwritten character image data into text, and it hinders the correct conversion of the handwritten character image data into text. When handwritten character image data is used as a learning target for AI, such information is removed. In other words, when handwritten character image data is used as a learning target for AI, a recognition accuracy improvement process is performed to remove digital information other than the digital information corresponding to the handwritten characters, thereby improving the recognition accuracy of the handwritten character image data.

[0012] Some documents, such as accounting slips, that are the target of data entry have fixed positions for input fields, and the input fields have pre-defined boxes for writing characters, so the inputter must write by hand by aligning the characters with the boxes. In such documents, by taking advantage of the fact that handwritten characters are written in line with the boxes, the handwritten characters can be divided into boxes and the AI ​​can learn each character individually.

[0013] In addition, there are cases where the items are "amount" or "date" and the characters entered are only numbers. By limiting the AI's learning targets to numbers only, it is possible to limit the targets to 10 types: "0, 1, 2, 3, 4, 5, 6, 7, 8, 9."

[0014] Taking advantage of the fact that handwritten characters are written in a grid, the handwritten characters were divided into grids, and the input characters were limited to items containing only numbers. The AI ​​was trained to learn each character individually, and the character recognition rate was 16.0%.

[0015] Furthermore, as shown in Figure 11, there are many lines and noise around handwritten characters. We assumed that these lines and noise were preventing the image data from being correctly matched to the text data, and after removing these lines and noise and training the AI, the character recognition rate was 89.7%.

[0016] Additionally, in cases where handwritten characters are entered into a grid, there may be some grids with no characters. These grids are excluded from AI processing. This reduces the workload of the AI. [Effects of the Invention]

[0017] Analog data consisting of handwritten characters can be converted into digital data with a high character recognition rate, which results in extremely high efficiency in inputting analog data consisting of handwritten characters, and a significant reduction in the time and effort required. [Brief explanation of the drawings]

[0018] [Figure 1] 1 shows a hardware configuration of an administrator terminal 1 according to a first embodiment. [Figure 2] 1 shows the configuration of an information communication network according to a first embodiment. [Figure 3] 1 shows the overall steps of the "item text data acquisition system used for verification" of the first embodiment. [Figure 4] 1 shows the item image data acquisition process (step 1) for the AI ​​processing target item in the first embodiment. [Figure 5] 10 shows the ruled line deletion process (step 2-1) of the first embodiment. [Figure 6] 10 shows the noise removal process (step 2-2) of the first embodiment. [Figure 7]10 shows a process of deleting blank space information (step 2-3) in the first embodiment. [Figure 8] 1 shows the process of removing blank single-character image data (step 2-4) and the process of obtaining single-character text data to be processed by AI (step 3-1) in the first embodiment. [Figure 9] 10 shows the AI-recognized text data acquisition process (step 3-2) of the first embodiment. [Figure 10] Item text data acquisition process used for verification in Example 1 (Step 4) [Figure 11] 1 shows image data after processing in Example 1 and image data without processing. [Explanation of symbols]

[0019] 1. System administrator terminal 2. Control section 3 CPU 4 ROM 5 RAM 6 Input Devices 7 Display device 8. Communication Control Device 9 Bus Lines 10 Storage device 11 Data storage unit 12 Program storage section DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. [Example]

[0021] Fig. 1 shows the hardware configuration of the administrator terminal 1. As shown in Fig. 1, the administrator terminal 1 is equipped with a control unit 2 for controlling the entire system. An input device 6, a display device 7, a communication control device 8, and a storage device 10 are connected to this control unit 2 via a bus line 9 such as a data bus.

[0022] The control unit 2 includes a CPU 3, a ROM 4, and a RAM 5. The CPU 3 performs various information processing and control operations in accordance with programs stored in various storage units, such as the ROM 4 and a storage device. The ROM 4 is a read-only memory that stores in advance various programs and data for the CPU 3 to perform various controls and calculations. The RAM 5 is a random access memory used by the CPU 3 as a working memory. Various areas can be allocated in the RAM 5 to perform various processes according to this embodiment.

[0023] The input device 6 is provided with a keyboard, a mouse, a touch panel, etc. (not shown). The keyboard is provided with various keys such as keys for inputting characters, a numeric keypad for inputting numbers, function keys for executing various functions, cursor keys, etc. The mouse is a pointing device, and is an input device that specifies a corresponding function by clicking on a key or icon displayed on the display device 7. The touch panel is an input device that is placed on the surface of the display device 7, and identifies a touch position of the user corresponding to various operation keys displayed on the screen of the display device 7, and accepts input of the operation keys displayed corresponding to the touch position.

[0024] A CRT, a liquid crystal display, etc. is used as the display device 7. This display device displays the results of inputs made through the keyboard or mouse, as well as image information.

[0025] The communication control device 8 is a control device for connecting the administrator terminal 1 to various external electronic devices such as other personal computers via a network. The communication control device 8 allows these various external electronic devices to access the administrator terminal 1, and allows search condition statements to be input from the external electronic devices.

[0026] The storage device 10 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. A hard disk is primarily used as the storage medium used in this storage device 10. The storage device 10 has a data storage unit 11, a program storage unit 12, and other storage units (not shown) (for example, storage units for backing up programs, data, etc. stored in this storage device 10). The data storage unit 11 stores data required by the system in this embodiment, as will be described later. The program storage unit 12 stores various processing programs in this embodiment, as will be described later.

[0027] The administrator terminal 1 of this embodiment can be configured not only by a computer system, but also by a LAN server, a computer communication host, a computer system connected to the Internet, etc. It is also possible to distribute functions to each device on the network and have the same configuration as the administrator terminal 1 over the entire network.

[0028] As shown in Figure 2, the verification system is composed of an administrator's computer, an entry worker's (first verification worker's) computer, and a verification worker's (second verification worker's) computer, all connected via an information and communication network such as a network or LAN. The verification system can be configured not only by computer systems, but also by LAN servers, computer communication hosts, computer systems connected to the Internet, etc.

[0029] As shown in FIG. 1, the storage device 10 includes a program storage unit 11 and a data storage unit 12 .

[0030] The program storage unit 11 stores programs for causing a computer to execute the following processes, all of which are processed by the CPU of the administrator's computer. (1) Acquisition of image data for items subject to AI processing (2) Deleting borders (3) Noise removal processing (4) Deleting white space (5) Single-space character image data exclusion processing (6) Acquisition of single character image data for AI processing (7) AI recognition text data acquisition processing (8) Item text data acquisition process for verification

[0031] The data storage unit 12 stores the following information in advance: (1) Location information of image data for each verification target (2) Information on items to be processed by AI (3) Item text data (4) Ruled line position information (5) Information to be processed for margins (6) AI-recognized text data (7) Other data

[0032] The data storage unit 12 records all information acquired during the execution of each program.

[0033] The terms used in this specification are defined as follows: "Entire image data" refers to image data (i.e., image data) obtained by digitizing the entire document including the items to be verified. An "item" is an item in the document to be verified that serves as a unit for verification, such as "date," "name," or "claim amount." "Items subject to AI processing" are items that have been listed in advance as items to be subjected to AI processing. "Item image data" is image data of an item (i.e., image data of an item). "Item text data" is text data of an item. "Item location information" is digital information that indicates the location of an item in a document. "One-character image data" is image data of one character (i.e., image data of one character). "One-character text data" refers to text data of one character. The "recognition accuracy improvement process" is a process for improving the accuracy when converting image data into text data. "Ruled line position information" is digital information that indicates the position of a ruled line in an item. "Character position information" is digital information that indicates the position of a character in an item. "Margin processing target information" is digital information that indicates the position of margins in an item. "AI processing" is a process that classifies single-character image data into single-character text data. AI-recognized text data is obtained through AI processing. "AI-recognized text data" is information on single-character image data of 10 types of numbers (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) and the text data corresponding to each single-character image data, and is data obtained through AI processing. "OCR processing" refers to processing using existing Optical Character Recognition. The "S" in each figure stands for step.

[0034] FIG. 3 is a flowchart showing the overall steps of the "system for acquiring item text data used for verification" of the first embodiment.

[0035] [Step 1] The CPU acquires and stores image data of the items to be processed by AI from the document to be verified. Details of step 1 are shown in steps 5 to 9 of Figure 4.

[0036] [Step 2] The CPU performs processing to improve recognition accuracy on the item image data. Specifically, it performs processing to delete ruled line information (step 2-1), noise (step 2-2), white space (step 2-3), and single-character blank image data (step 2-4). The programs for each processing are recorded in the data storage unit. The processing to delete ruled line information and noise is always performed in this order: ruled line information deletion, then noise deletion. White space deletion is performed only when white space is present. After the processing to improve recognition accuracy is performed, each data is saved. Details of step 2 are shown in steps 10 to 22 of Figures 5 to 8.

[0037] [Step 3] The CPU processes the item text data that has undergone the recognition accuracy improvement process and is then responsible for acquiring and saving the AI-recognized text data, as shown in steps 17 in Figure 8 to 24 in Figure 9.

[0038] [Step 4] The CPU acquires and stores the item text data used for verification, as shown in step 25 of Fig. 10. In the verification process, the first worker acquires the primary text data using the item text data used for verification.

[0039] Figure 4 shows a flowchart of each step in the process of acquiring item image data for items subject to AI processing (Step 1).

[0040] [Step 5] The CPU uses a scanner to digitize the analog data of the entire document to be verified (for example, the entire accounting slip) and obtains the entire image data (i.e., the image data obtained by digitizing the entire document including the items to be verified). In step 5, the CPU obtains the entire image data and stores it in the data storage unit together with the document information to be verified.

[0041] [Step 6] The document to be verified contains items such as date, name, and billing amount (i.e., items that serve as units for verification) depending on the content of the document. Each item is located at a fixed position within the document to be verified, meaning that the document to be verified is standardized. Therefore, the position information of each item image data is obtained in advance and recorded in the data storage unit for each document to be verified. The CPU can obtain the position information of the desired item image data from the document information of the verification target obtained in step 5 and the position information of the item image data for each verification target recorded in advance. The CPU acquires the desired item image data from the overall image data based on the position information of the item image data acquired in this way, and stores the acquired item image data in the data storage unit.

[0042] [Step 7] Items that meet the following conditions: (1) the characters used in the item are numbers, and (2) the characters written in the item can be separated into individual characters are considered to be items that should be subject to AI processing, and the items are recorded in advance in the data storage unit as "information on items subject to AI processing." The CPU acquires the "item information" and "AI processing target item information" and determines whether the item is a target for AI processing.

[0043] [Step 8] If the item is not subject to AI processing, the CPU obtains and stores item text data (i.e., text data of the item) from the item image data (i.e., image data of the item) through OCR processing, and then ends step 1.

[0044] [Step 9] If the item is subject to AI processing, the CPU performs step 2-1 [deletion process of ruled line information] and ends step 1.

[0045] FIG. 5 is a flowchart showing each step in the ruled line deletion process (step 2-1).

[0046] [Step 10] The CPU obtains ruled line position information (that is, digital information indicating the position of a ruled line in an item) from the data storage unit.

[0047] [Step 11] The CPU replaces the pixels at the ruled line positions with white pixels, and stores the replaced item image data in the data storage unit.

[0048] [Step 12] The CPU starts step 2-2 [noise removal process] and ends step 2-1 [ruled line removal process].

[0049] FIG. 6 is a flowchart showing each step in the noise removal process (step 2-2).

[0050] [Step 13] The CPU applies a Gaussian blur to each pixel of the item image data.

[0051] [Step 14] The CPU replaces pixels below a predetermined value (for example, below 150 levels in a grayscale image) with white pixels, stores the item image data after replacement in the data storage unit, and ends the process.

[0052] FIG. 7 is a flowchart showing each step in the process of deleting blank space information (step 2-3). For example, if the space allocated to each character in an item is long vertically, there will be a blank space above the space allocated to each character. The blank space here refers to such a blank space. The blank space locations are known in advance for each item. The location of the blank space within the item is recorded in advance for each item as blank space location information.

[0053] [Step 15] The CPU selects a margin processing target. An item with margin position information is the target for margin processing. The CPU acquires margin position information for the margin processing target (i.e., digital information indicating the position of the margin in the item).

[0054] [Step 16] The CPU deletes the pixels in the margins, stores the deleted item image data in the data storage unit, and ends the process.

[0055] Figure 8 shows the process of removing one-character image data from a blank space (step 2-4), and Figure 8 also shows a flowchart of each step in the process of obtaining one-character text data by AI processing (step 3-1).

[0056] [Step 17] The CPU acquires the ruled line position information from the data storage unit. The CPU divides the item image data at the ruled line positions to acquire single character image data. The CPU stores the position information of the single character image data in the item image data together with the single character image data.

[0057] [Step 18] The CPU counts the number of pixels that are black for each character image data.

[0058] [Step 19] The CPU determines whether the number of black pixels in the single character image data is equal to or greater than a predetermined number (e.g., 7% or more of the number of image pixels). If the number of black pixels in the single character image data is equal to or greater than the predetermined number, it can be determined that a character exists in the single character image data. If the number is less than the predetermined number, it can be determined that a character does not exist in the single character image data.

[0059] [Step 20] If the number of black pixels in the single-character image data is less than the predetermined number in step 19, the CPU excludes the single-character image data from the AI ​​processing target and terminates the blank single-character image data exclusion process (step 2-4). This is because the single-character image data is determined to be blank (no character). By excluding blank single-character image data from the AI ​​processing target, meaningless AI processing can be eliminated.

[0060] [Step 21] If the number of black pixels in the one-character image data is equal to or greater than the predetermined number in step 19, the CPU determines that one-character image data as one-character image data to be subjected to AI processing.

[0061] [Step 22] The CPU starts step 3-2 [AI processing] for the single-character image data to be AI processed, and ends step 3-1 "processing for obtaining single-character text data by AI processing."

[0062] FIG. 9 is a flowchart showing each step in the AI-recognition text data acquisition process (step 3-2).

[0063] [Step 23] The big data of combinations of single-character image data and corresponding text data is processed by AI, and the single-character image data is classified. As a result of the AI ​​processing, AI-recognized text data (i.e., single-character image data of 10 types of numbers (0, 1, 2, 3, 4, 5, 6, 7, 8, 9) and information on the text data corresponding to each single-character image data) is obtained.

[0064] [Step 24] The AI-recognized text data obtained in step 23 is stored in the data storage unit.

[0065] FIG. 10 is a flowchart showing each step in the process of obtaining item text data used for verification.

[0066] [Step 25] Based on the positional information of each character in the "single-character image data," the CPU arranges each piece of AI-recognized text data so that it matches the positional relationship of each corresponding "single-character image data." The AI-recognized text data arranged in this way is saved and used as item text data for verification by the first worker. The process ends.

Claims

1. A verification system for acquiring text data corresponding to characters of analog data from characters of analog data, comprising: The characters in the analog data are handwritten, a primary text data acquisition means for acquiring primary text data recorded by a primary worker; a secondary text data acquisition means for acquiring secondary text data recorded by a secondary worker; an AI-recognized text data acquisition means for classifying handwritten character image data, which is data obtained by digitizing analog data of handwritten characters, and acquiring AI-recognized text data, which is data recognized as text data corresponding to the handwritten character image data; The primary text data acquisition means The handwritten character image data is compared with the AI-recognized text data; Among the AI-recognized text data, the data determined to match the handwritten character image data is acquired as primary text data. Among the AI-recognized text data, the data determined not to match the handwritten character image data is corrected to match the handwritten character image data and acquired as primary text data; The secondary text data acquisition means The handwritten character image data; the primary text data is compared; acquiring, as secondary text data, the primary text data that is determined to match the handwritten character image data; Any of the primary text data determined not to match the handwritten character image data is corrected to match the handwritten character image data and acquired as secondary text data; The AI ​​recognition text data acquisition means AI-recognized text data is acquired by an AI function that classifies the handwritten character image data based on a combination of the handwritten character image data and text data corresponding to the handwritten character image data; The AI ​​function is a process that removes digital information that exists around handwritten character image data but is not digital information that corresponds to handwritten characters, and is a verification system that classifies data that has undergone recognition accuracy improvement processing to improve the recognition accuracy of handwritten character image data.

2. In the AI ​​recognition text data acquisition means, One of the characters constituting the handwritten characters, handwritten character image data of one character obtained by dividing the handwritten character image data; Based on a combination of text data corresponding to the handwritten character image data of the single character, AI function that classifies image data of handwritten characters The verification system according to claim 1 , wherein AI-recognized text data corresponding to one character is acquired.

3. In the AI ​​recognition text data acquisition means, Handwritten character image data for single-character numbers and Based on a combination of text data corresponding to the handwritten character image data for the single number character, By using AI technology to classify handwritten image data of single-character numbers, The verification system according to claim 1, wherein AI-recognized text data for a single-character number is acquired.

4. In the first text data acquisition means, The comparison between the handwritten character image data and the AI-recognized text data is Based on the position information of each character in the handwritten character image data 2. The verification system according to claim 1, wherein AI-recognized text data corresponding to each character is arranged.

5. In the recognition accuracy improvement process, Digital information other than digital information equivalent to handwritten characters For information with location information, 2. The verifying system according to claim 1, wherein pixels in locations where digital information other than digital information corresponding to handwritten characters exists are replaced with white pixels based on the position information.

Images