Image processing system, device, method, and program

Abstract

To simply make a correction when a character string resulting from a character recognition in an area selected by a user is not within a desired range.SOLUTION: In the present invention, an image processing system executes character recognition processing on a document image, and specifies a candidate division point in a character string resulting from the recognition in the character recognition processing. When a desired position is designated by a user on the displayed document, the image processing system determines a character string corresponding to the designated position as a character string to be output, and displays the candidate division point. When the candidate division point is operated by the user, the image processing system changes the character string to be output to a character string obtained through division based on the candidate division point.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to an image processing system, an apparatus, a method, and a program. [Background technology]

[0002] There is a business that scans paper documents, digitizes them, and stores them. Conventionally, there has been a system that performs character recognition when digitizing documents and uses the results as file names. For example, there is a system in which a user selects the character recognition results from a document image and saves the character recognition results as file names in any storage. However, since the character recognition results are used, there is a tendency for the character recognition results to fluctuate. For example, when an extra blank character is present in a character string to be set as a file name, the file name also contains blank characters, which is not preferable. Therefore, Patent Document 1 discloses a method of converting the character recognition results into a character string suitable for a file name, such as by removing leading blank characters, in order to use the character recognition results as file names. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-74609 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the method of Patent Document 1, when the selected character recognition contains a blank character, it is possible to convert it into a character string suitable for a file name. However, depending on the user, the range of the character string selected as the file name as a result of character recognition may not be desirable. In addition, when giving file names to similar document images, the range of character strings desired as file names may differ from user to user, so it is difficult to determine the character string to be used as the file name based on a single standard. For example, when using a date written in a document image as the file name, some users may want to use the character string of the item name associated with the date (e.g., "payment due date") together with the file name, while other users may want to use only the date as the file name. [Means for solving the problem]

[0005] The present invention Terminal equipment teeth, The document was scanned and obtained Document Images a first display control means for controlling to display a UI including the document image, and a frame representing an area identified as an area including characters by an analysis process of the document image, the frame representing an area corresponding to a position specified by a user on the document image, the area including characters being displayed; Against do Character Recognition Processing By recognition Will be String The first String Display in the display area of ​​the UI , An item for accepting user operations of In the above frame View The second control display control Means, a receiving means for receiving a user operation to move the item displayed in the frame; The above Move an item The above user The Misao Made by Gana If so, Displayed in the display area String , from the first string: The above User Operation Based on A second character string that is a part of the first character string, Change to String Display do The third display control Means and The file name of the file is set using the character string displayed in the display area. It is characterized by: Effect of the Invention

[0006] According to the present invention, it is possible to provide operability that allows a user to easily correct a character string resulting from character recognition in an area selected by the user when the character string is not within a desired range. [Brief description of the drawings]

[0007] [Figure 1]FIG. 1 is a diagram illustrating a system configuration of an image processing system. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of an image forming apparatus 101. [Diagram 3] 2 is a diagram illustrating the hardware configuration of an image processing server 102 and a user terminal 103. FIG. [Figure 4] FIG. 4 is a diagram showing an example of a form image 400 and its character recognition result. [Diagram 5] FIG. 2 is a diagram illustrating a processing flow according to the first embodiment. [Figure 6] FIG. 13 is a diagram showing a processing flow of text segmentation. [Figure 7] FIG. 13 is a diagram showing a process flow of candidate division. [Figure 8] FIG. 13 is a diagram showing a process flow of text correction. [Figure 9] FIG. 13 is a diagram showing a list of regular expression definitions. [Figure 10] 11 is a table showing an example of a character recognition result. [Figure 11] 13 is an example showing the position of a text segmentation result. [Figure 12] 13 is an example showing the positions of candidate division results. [Figure 13] 13 is an example of a text correction result. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment does not limit the present invention, and all of the configurations described in the embodiment are not necessarily essential means for solving the problems of the present invention.

[0009] <First embodiment> 1 is a diagram showing an example of the configuration of an image processing system 100 according to the first embodiment. The image processing system 100 includes an image forming apparatus 101, an image processing server 102, and a user terminal 103. The image forming apparatus 101, the image processing server 102, and the user terminal 103 are connected to each other via a network 104 and are capable of communicating with each other.

[0010] The image forming apparatus 101 is a multifunction machine capable of receiving a print request (print data) of image data from the user terminal 103 and printing the image data, reading the image data with a scanner provided in the image forming apparatus 101, and printing the image data read with the scanner. The image processing server 102 is an image processing apparatus capable of executing image processing, which will be described later, on the image data read with the scanner of the image forming apparatus 101 and transmitting the image processing result to the user terminal 103. The image processing server 102 may be a cloud, that is, a virtual server disposed on the Internet. The user terminal 103 is capable of performing additional processing of the image processing result received from the image processing server 102 interactively with a user using an application having a user interface. In this embodiment, the user terminal 103 is assumed to be a general PC equipped with a display, a keyboard, and a mouse, but may be, for example, a mobile terminal equipped with a touch panel.

[0011] In this embodiment, a series of data input support processes will be described, in which an image forming device 101 scans paper documents such as invoices, an image processing server 102 extracts the necessary information from the documents and stores it electronically, and a user terminal 103 provides a user interface that allows the user to check and correct the extracted results.

[0012] 2 is a diagram showing an example of the configuration of the image forming apparatus 101. The image forming apparatus 101 has a controller 201, a printer 202, a scanner 203, and an operation unit 204. The controller 201 has a CPU 211, a RAM 212, a HDD 213, a network I / F 214, a printer I / F 215, a scanner I / F 216, an operation unit I / F 217, and an expansion I / F 218.

[0013] The CPU 211 controls the entire image forming apparatus 101. The CPU 211 can control data transmission and reception with the RAM 212, the HDD 213, the network I / F 214, the printer I / F 215, the scanner I / F 216, the operation unit I / F 217, and the expansion I / F 218. The CPU 211 also loads a control program (instruction) read from the HDD 213 in the RAM 212 and executes the instruction loaded in the RAM 212. The HDD 213 stores a control program executable by the CPU 211, setting values ​​used in the image forming apparatus 101, data related to processing requested by a user, and the like. The RAM 212 has an area for temporarily storing instructions read by the CPU 211 from the HDD 213. The RAM 212 can also store various data required for executing instructions. For example, in image processing, the CPU 211 can perform processing by loading input data in the RAM 212.

[0014] The network I / F 214 is an interface for performing network communication with devices in the image processing system 100. The network I / F 214 can notify the CPU 211 of data reception and can transmit data on the RAM 212 to the network 104. The printer I / F 215 can transmit print data transmitted from the CPU 211 to the printer 202 and can transmit the printer status received from the printer 202 to the CPU 211. The scanner I / F 216 can transmit an image reading instruction transmitted from the CPU 211 to the scanner 203, transmit image data received from the scanner 203 to the CPU 211, and can transmit the status received from the scanner 203 to the CPU 211. The operation unit I / F 217 can transmit an instruction from a user input from the operation unit 204 to the CPU 211 and can transmit screen information for the user to operate to the operation unit 204. The expansion I / F 218 is an interface that enables an external device to be connected to the image forming apparatus 101. The expansion I / F 218 includes, for example, a USB (Universal Serial Bus) type interface. When an external storage device such as a USB memory is connected to the expansion I / F 218, the image forming apparatus 101 can read data stored in the external storage device and write data to the external storage device.

[0015] The printer 202 is capable of printing image data received from the printer I / F 215 on paper, and of transmitting the status of the printer 202 to the printer I / F 215 .

[0016] The scanner 203 can read information displayed on a paper sheet placed on a reading unit, digitize the information, and transmit the digitalized information to the scanner I / F 216 in accordance with an image reading instruction received from the scanner I / F 216. The scanner 203 can also transmit its own status to the scanner I / F 216.

[0017] The operation unit 204 is an interface that allows a user to perform operations for issuing various instructions to the image forming apparatus 101. For example, the operation unit 204 is equipped with a liquid crystal display having a touch panel, and provides an operation screen to the user of the image forming apparatus 101 and accepts operations from the user. Details of the operation unit 204 will be described later with reference to FIG.

[0018] 3(a) is a diagram showing an example of the configuration of the image processing server 102. The image processing server 102 has a CPU 301, a RAM 302, a HDD 303, and a network I / F 304. The CPU 301 controls the entire image processing server 102. The CPU 301 can control the exchange of data with the RAM 302, the HDD 303, and the network I / F 304. The CPU 301 also loads a control program (instructions) read from the HDD 303 into the RAM 302, and executes the instructions loaded into the RAM 302.

[0019] FIG. 3B is a diagram showing an example of the configuration of the user terminal 103. The user terminal 103 includes a CPU 311, a RAM 312, a HDD 313, a network I / F 314, and an input / output I / F 315. The CPU 311 controls the entire user terminal 103. The CPU 311 can control data transmission and reception between the RAM 312, the HDD 313, the network I / F 314, and the input / output I / F 315. The display 320 is configured with a display device such as a liquid crystal display, and displays display information received from the input / output I / F 315. The input device 330 is configured with a pointing device such as a mouse or a touch panel, and a keyboard, and receives an operation from a user and transmits operation information to the input / output I / F 315. The HDD 313 can store image processing results received from the image processing server 102 via the network I / F 314. In this embodiment, the CPU 311 loads an application program read from the HDD 313 into the RAM 312, and causes the operation unit I / F 315 to display display information and accept user operations.

[0020] FIG. 4(a) is a diagram showing an example of a form image 400 assumed in this embodiment. The form image 400 is an image acquired by reading a paper document (e.g., an invoice) with the scanner of the image forming apparatus 101. Item values ​​401 to 403 are examples of item character strings to be extracted by the image processing system 100. The item value 401 in FIG. 4(a) is a title value indicating the contents of this document, the item value 402 is a date value indicating the issue date, and the item value 403 is a value of the billing amount. Note that in the example of FIG. 4(a), the item values ​​401 to 403 are enclosed in rectangular frames to indicate their positions, but no rectangular frames are drawn on the form image acquired by scanning.

[0021] FIG. 4B shows an example of a character string (OCR character string) that is a result of character recognition obtained when a general-purpose region analysis process and an optical character recognition (OCR) process are performed on a form image 400. Eight character regions, namely, character strings 410 to 417, are identified, and an OCR character string is extracted from each character region. In FIG. 4B, a rectangular frame indicates a position corresponding to each character region extracted based on the results of the region analysis process and the OCR process. A character string 410 is obtained as a character string corresponding to one character region that includes the character string of the item value 401 and a character string to the left of it. A character string 411 is extracted as a character string corresponding to one character region that includes the item value 402 and a character string to the left of it. A character string 413 is also extracted as a character string corresponding to a region that includes the item value 403 and a character string to the left of it.

[0022] We will explain a use case where this character string is used in a UI for creating a file name by the user. For example, we will explain a UI (user interface) where, when a user clicks a desired position on a document image, a character area corresponding to the clicked position is selected based on the area analysis result in Figure 4(b).

[0023] In such a UI, when a user clicks on the character string "ABC" to specify it, the character string in the character region based on the region analysis result (i.e., the character string "Invoice for ABC Co., Ltd." in character region 410) is selected. Therefore, if that user wants to select only the "ABC Co., Ltd." portion as the file name, they must perform an operation to delete the redundant "Invoice" character string from the selected character string. On the other hand, if another user who wants to add "ABC Co., Ltd." to the file name is operating, they must perform an operation to delete the redundant "Invoice" character string from the character string "Invoice for ABC Co., Ltd." specified by clicking.

[0024] The following describes a correction operation UI that allows a user to easily correct a character string resulting from character recognition in a system that assigns a file name based on a character string resulting from character recognition of a character area corresponding to a position specified by the user when the character string is not the character string desired by the user.

[0025] Before describing the processing flow of this embodiment, the regular expression definition list in FIG. 9 will be described first.

[0026] A regular expression definition list 900 in FIG. 9 is an example showing a plurality of regular expression definitions in a table format used in the text segmentation process in step S503 described later. In the regular expression definition list 900, a combination of a regular expression and a regular expression parameter is associated with each definition ID to define it. A plurality of regular expression definitions predefined in this list are stored in the HDD 303 of the image processing server 102. A regular expression is a description of a character string to be extracted, such as an item to be extracted, for example, a date, a telephone number, an amount, or a character included in a document title, in one regular expression. A regular expression parameter is a parameter defined for each regular expression, which describes how to interpret an OCR character string to be a target when performing a regular expression search. For example, the parameter describes how far apart adjacent characters must be to be treated as a space character (blank character).

[0027] In the example of the regular expression definition list 900 in FIG. 9, three regular expression definitions 910, 920, and 930 are defined.

[0028] The regular expression definition ID910 consists of a regular expression “￥S*書” and a regular expression parameter “スペース=2h”. The regular expression “￥S*書” represents a pattern combining multiple characters other than the space character (￥S) and the character “書”. For example, strings such as “請求書” and “見積書” can be searched as corresponding patterns. The regular expression parameter “スペース=2h” indicates that when converting the OCR character string into a search string, if the distance between adjacent characters is more than twice the character height (h), a space character is inserted for handling. In this embodiment, the character height is used to define the threshold for treating as a space character in the regular expression parameter. However, for example, the pixel size of the image, the physical distance on the paper surface, the average character width, etc. may also be used as a reference.

[0029] The regular expression definition 920 is a regular expression definition related to the date, consisting of a regular expression “￥d{2,4}年￥d{1,2}月￥d{1,2}日” and a regular expression parameter “スペース削除”. The regular expression “￥d{2,4}年￥d{1,2}月￥d{1,2}日” represents a pattern combining 2 - 4 digits, “年”, 1 - 2 digits, “月”, 1 - 2 digits, and “日”. A date string that matches this pattern can be searched. The regular expression parameter “スペース削除” indicates that when converting the OCR character string into a search string, regardless of the distance between adjacent characters, no space character is inserted.

[0030] Regular expression definition 930 consists of a regular expression of "[1-9][￥d,]*yen" and a regular expression parameter of "space=1h". The regular expression of "[1-9][￥d,]*yen" represents a pattern that combines a number starting with any one of the digits 1 to 9 and including one or more commas with "yen", and it is possible to search for a character string that represents an amount of money that matches this pattern. The regular expression parameter "space=1h" indicates that when converting an OCR character string into a search character string, if the distance between adjacent characters is equal to or greater than one character height, based on the character height (h), a space character is inserted.

[0031] Regular expression definition 940 is composed of a regular expression of "\s" and a regular expression parameter of "space=3.5h". Regular expression definition 940 represents a pattern of a space character (\s), and is capable of searching for a character string of space characters. The regular expression parameter "space=3.5h" indicates that when converting text information into a search character string, if the distance between adjacent characters is 3.5 characters or more based on the character height, a space character is inserted and handled. In other words, this regular expression definition 940 is a pattern description in which if the distance between characters is 3.5 times the character height or more, a space character is inserted between the characters, and the space character matches the regular expression.

[0032] Regular expression definition list 901 in Fig. 9 is an example showing one or more regular expression definitions in table format used in the candidate division process in step S504 described later. Regular expression definition list 901 has the same format as regular expression definition list 900, and defines each definition ID by associating a combination of a regular expression and a regular expression parameter. In the example of regular expression definition list 901 in Fig. 9, one regular expression definition 950 is defined.

[0033] Regular expression definition ID 950 consists of a regular expression of "\s" and a regular expression parameter of "space=0.5h". Regular expression definition 950 represents a pattern of a space character (\s), and is capable of searching for a character string of space characters. The regular expression parameter "space=0.5h" indicates that when converting character string information resulting from text segmentation into a search string, if the distance between adjacent characters is 0.5 times the character height (h) or more, a space character is inserted. In other words, this regular expression definition 950 is a pattern description in which if the distance is 0.5 times the character height or more, a space character is inserted and the space character matches the regular expression.

[0034] 9 is an example of a table showing a plurality of regular expression definitions used in the text correction process in step S505, which will be described later. The regular expression definition list 902 defines each definition ID by associating a regular expression, a regular expression parameter, and a process to be executed on text information that matches the regular expression. This definition list is stored in the HDD 303 of the image processing server 102.

[0035] A regular expression similar to that of regular expression definition 920 and regular expression parameters are associated with definition ID 960 of the regular expression definition, and further, the processing executed when the regular expression matches is to remove space characters from the matched text information.

[0036] A regular expression similar to that of regular expression definition 930 and regular expression parameters are associated with definition ID 970 of the regular expression definition, and further, the processing executed when the regular expression matches is to delete "," from the matching text information.

[0037] Definition ID 980 of the regular expression definition is associated with a regular expression similar to that of regular expression definition 930 and regular expression parameters, and further, the processing executed when the regular expression matches is to delete the “circle” from the matching text information.

[0038] Using the form image 400 in FIG. 4 and the regular expression definition lists 900, 901, and 902 in FIG. 9 as examples, the image processing of this embodiment will be described with reference to the flowcharts in FIGS.

[0039] 5, the CPU 211 of the image forming apparatus 101 transmits the form image 400 read by the scanner 203 to the image processing server 102. The image processing server 102 acquires the form image 400 transmitted from the image forming apparatus 101.

[0040] Next, in S502, the CPU 301 of the image processing server 102 performs region analysis processing on the form image 400 to identify character regions, and executes character recognition processing on the character regions. As a result of the character recognition processing, the CPU 301 obtains the coordinates of the character region (character block), the coordinates of each character in the character region, and the character code of the character recognition result. The array of character codes obtained in this manner for each character region is called an OCR character string (character string as a result of character recognition). It is assumed that character strings 410 to 417 are obtained as OCR character strings as a result of performing character recognition processing on the form image 400.

[0041] Next, in S503, the CPU 301 of the image processing server 102 performs text segmentation processing. Details of this text segmentation processing will be described with reference to the flowchart of FIG.

[0042] In S601 in FIG. 6, the CPU 301 of the image processing server 102 selects one of the regular expression definitions (eg, the regular expression definition 910) from the regular expression definition list 900 in FIG. 9 stored in the HDD 303 as the processing target.

[0043] Next, in S602, the CPU 301 of the image processing server 102 interprets the character string resulting from the character recognition obtained in S502 based on the regular expression parameters of the regular expression definition processed in S601, and normalizes it as a search character string.

[0044] Figure 10 shows an example of a character recognition result. The character recognition result 1001 is the character recognition result of the character string 410. The character recognition result 1002 is the character recognition result of the character string 411. Also, the character recognition result 1003 is the character recognition result of the character string 413. In each table of the character recognition results 1001 to 1003, the rows of characters represent each recognized character, and the rows of distances represent the distances to the next character, with the character height as the relative reference. The regular expression parameter of the regular expression definition 910 is "space = 2h", which indicates that if the distance between characters is more than two times the character height with the character height as the relative reference, it is regarded as a space character. In the recognition result 1001, since the character "様" is separated from the adjacent character "請" by a distance corresponding to 2.1 character heights, a space character is inserted here to generate the search string "ABC(株)様 請求書".

[0045] Note that for each regular expression parameter, the search string changes. For example, if it is defined as "space = 1h", space characters are further inserted between "請" and "求", and between "求" and "書", resulting in "ABC(株) 様 請 求 書", and if it is defined as "space deletion", it becomes "ABC(株)様請求書".

[0046] Similarly, the process of S602 is also executed for the remaining character recognition results 411 to 417 to generate search strings for all character strings.

[0047] Next, in S603, the CPU 301 of the image processing server 102 performs a regular expression search to determine whether the search string obtained in S602 matches the regular expression of the regular expression definition processed in S601.

[0048] When the regular expression definition 910 is searched for the search string "ABC Co., Ltd. Invoice" in string 410, the "invoice" portion matches. Next, when the regular expression definition 910 is searched for the search string "Issue date: May 15, 2020" in string 411, no matches are found. Similarly, the same process is performed on the remaining strings 412 to 417 using the regular expression definition 910, and as a result, the regular expression does not match the other strings.

[0049] Next, in S604, the CPU 301 of the image processing server 102 stores in the RAM 302 the matching information for “invoice” obtained in the search result of S603.

[0050] Next, in S605, the CPU 301 of the image processing server 102 determines whether any unprocessed regular expression definitions remain, and if any unprocessed regular expression definitions remain, returns to S601 and repeats the same processing with one of the unprocessed regular expression definitions as the next processing target.

[0051] For example, if regular expression definition 910 is the first processing target, regular expression definition 920 is the next processing target. In this case, in S601, a search string is generated for character recognition result 1002 based on the parameters of regular expression definition 920. Since the parameter of regular expression definition 920 is "delete spaces," no space characters are inserted regardless of the distance between characters, and therefore "Publication date: May 15, 2020" is obtained as a search string from character recognition result 1002. Then, the search result of "May 15, 2020" is obtained as a portion that matches the regular expression of regular expression definition 920.

[0052] Similarly, when the regular expression definition 930 is the processing target, in S602, a search character string of "Total amount: 11,286 yen" is formed for the character recognition result 1003 based on the parameter "space=1h" of the regular expression definition 930. Then, in S603, "11,286 yen" is searched for as a portion that matches the regular expression of the regular expression definition 630.

[0053] In S606, the CPU 301 of the image processing server 102 performs character string division processing based on the search results stored in the RAM in the processing of S604. The division processing is a processing of dividing the OCR character string at both ends of the part in the OCR character string that matches the regular expression. For example, in the OCR character string 410 "ABC (Stock) Invoice", the left and right of "Invoice" are used as character string separators. However, since the right side of "Invoice" is the right end of the OCR character string, division does not occur, and the OCR character string 410 is divided at the left position of "Invoice" (i.e., between "Mr." and "Request"), and the OCR character string 410 is divided into two character strings. Similarly, processing is performed on "May 15, 2020" and "11,286 yen", and the processing of the flowchart in FIG. 6 is terminated. The character string after being divided by the text division processing is referred to as a text division result. This text division result includes text information indicating the character string after division and character position information of the circumscribing rectangle of each character.

[0054] Fig. 11 is a diagram showing the text segmentation results after applying the text segmentation process described in detail in Fig. 6 to the form image 400. Character string 410 of the character recognition result is segmented into text segmentation results 1100 and 1101, character string 411 of the character recognition result is segmented into text segmentation results 1102 and 1103, and character string 413 of the character recognition result is segmented into text segmentation results 1104 and 1105. Note that character recognition results 412, 414 to 417 remain original.

[0055] Next, in S504, the CPU 301 of the image processing server 102 performs candidate division processing. Details of this candidate division processing will be described with reference to the flowchart of FIG.

[0056] In S701 of FIG. 7, the CPU 301 of the image processing server 102 selects one of the regular expression definitions (regular expression definition 950) from the regular expression definition list 901 stored in the HDD 303 as a processing target. Then, by executing the processes of S702 to S705, it is determined whether or not there is a pattern matching the regular expression definition selected as the processing target in the text segmentation result segmented in the text segmentation process of S503. The processes of S702 to S705 are similar to the processes of S602 to S605, and detailed explanations are omitted. In the example of the regular expression definition list 901 of FIG. 9, since only one regular expression definition 950 is defined, the position of the space character inserted in S702 is searched for in S703, and the position of the searched space character is stored as a matching position in S704. The stored position information of the space character is used as the position of a candidate division point in the UI of FIG. 12 displayed in S506 described later.

[0057] 9 defines only the regular expression definition 950 for identifying the position of a space character, but the invention is not limited to this. For example, a regular expression may be defined so that the position of a ":" (colon) or a ";" (semicolon) can also be searched for. When searching for the position of a ":" (colon) or a ";" (semicolon), there is no need to insert a space character, so the regular expression parameter may be set to delete spaces.

[0058] Next, in S505, the CPU 301 of the image processing server 102 performs text correction processing. Details of this text correction processing will be described with reference to the flowchart of FIG.

[0059] 8, the CPU 301 of the image processing server 102 selects one of the regular expression definitions from the regular expression definition list 902 stored in the HDD 303 as the processing target. For example, the processing of S802 to S807 is repeated for each of the regular expression definitions 960, 970, and 980 in this order. The processing of S802 to S803 is similar to the processing of S602 to S603, and therefore will not be described in detail. However, it is determined whether or not a pattern matching the regular expression definition selected as the processing target is present in the text segmentation result obtained by the text segmentation processing of S503.

[0060] 13 is a character string determined to match regular expression definition 960 from among the character strings resulting from the text segmentation. The character rows in the table of text segmentation result 1301 represent each recognized character, and the distance rows represent the relative character height distance to the next character. Also, text segmentation result 1302 is a character string determined to match regular expression definition 970 and regular expression definition 980 from among the character strings resulting from the text segmentation.

[0061] In S804, the CPU 301 of the image processing server 102 inserts space characters into the text information of the character recognition result and the text information of the text segmentation result, using a predefined distance between characters. In this embodiment, the space characters are inserted with "space=0.5h". For example, when the space insertion process of S804 is performed on the text segmentation result 1301, the space insertion result 1303 is obtained. Also, when the space insertion process of S804 is performed on the text segmentation result 1302, the space character is not inserted, and the result is the space insertion result 1304.

[0062] In S805, the CPU 301 of the image processing server 102 proceeds to the process of S806 for the character strings determined in S803 to match the regular expression definition. Since the text segmentation result 1301 and the text segmentation result 1302 are matching character strings, they are processed in S806. Note that for character strings not determined in S803 to match the regular expression definition, they are not processed in S806 and proceed to S807.

[0063] In S806, the CPU 301 of the image processing server 102 executes the process associated with the regular expression definition of the processing target. For the text segmentation result 1301 that matches the regular expression definition 960, although a space character was inserted in S804 to form the character string 1303, in S806, since the process associated with the regular expression definition 960 is a process of removing the space character, as a result, the text correction result 1305 is obtained. Also, for the text segmentation result 1302 that matches the regular expression definition 970, for the character string 1304 after the process of S804, the process associated with the regular expression definition 970 (the process of removing ",") is executed to obtain the text correction result 1306. Furthermore, since the text segmentation result 1302 also matches the regular expression definition 980, for the text correction result 1306, the process associated with the regular expression definition 980 (the process of removing "yen") is further executed to obtain the text correction result 1307.

[0064] In S506, the CPU 301 of the image processing server 102 transmits information for causing the user terminal 103 to display a UI screen for assigning a file name. The information to be transmitted includes a document image for display, a character string of the character recognition result of each character area, each character area position information, position information of candidate segmentation points, and the like. The CPU 311 of the user terminal 103 displays the document image on the display 320 based on the received information, and when the user designates a desired position on the document image, performs a UI display for assigning a file name based on the character string corresponding to the designated position. The display of the UI screen may be a web application displayed via a web browser provided in the user terminal 103, or may be displayed using a dedicated application.

[0065] FIG. 12 is a diagram showing the correspondence between the positions of the character strings resulting from the text segmentation process in S503 and the positions of the candidate segmentation points resulting from the candidate segmentation process in S504 and the positions on the document image. The positions of the character strings resulting from the text segmentation process in S503 are shown as text segmentation results 1100 to 1105, as in FIG. 11. The positions of the results of the candidate segmentation process in S504 are shown as candidate segmentation points 1200 to 1203. The candidate segmentation point 1200 shows the position of the space character matched in the process of S703 in the text segmentation result 1100 as a candidate segmentation point. The candidate segmentation point 1201 and the segmentation point 1202 show the positions matched in the process of S703 in the text segmentation result 1103. The candidate segmentation point 1203 shows the position of the space character matched in the process of S703 in the character string 417 resulting from the character recognition as a candidate segmentation point.

[0066] In the file name assigning UI screen displayed in S506, when the user specifies a desired position on the document image, the region of the character string corresponding to the specified position (any of 412 to 417, 1100 to 1105 in FIG. 12) is focused, and the recognition result of the character string is input in the file name input field. The candidate division points 1200 to 1203 are usually hidden, but when candidate division points are set for the focused region, the positions of the candidate division points are displayed at the time of the focus so that the user can select the candidate division point. For example, when the position specified by the user corresponds to the text division result 1100, the region of the text division result 1100 is displayed in focus, and the candidate division point 1200 is displayed so that it can be specified. The positions of the candidate division points may be displayed as triangular marks as shown in FIG. 12, or may be displayed as other marks such as vertical bars.

[0067] Then, in S507, the CPU 311 of the user terminal 103 changes the character string used as the file name by modifying the character string already entered in the file name input field, triggered by the user's operation on the candidate division point. For example, if the character string "ABC (Stock)" in the area focused on by the user's click operation on the document image is not the character string desired by the user, the user can further perform an operation of pressing the candidate division point 1200 to modify the text division result and obtain "ABC (Stock)" as the output result. Also, if the text division result 1103 "May 15, 2020" is not the output result desired by the user, the text division result can be modified by pressing the division point 1202 to obtain "May 2020" as the output text.

[0068] In this embodiment, when a user clicks (or touches) a candidate division point, the character string to the left of the candidate division point is output, but this is not limited to this. For example, when a user presses a candidate division point and drags it to the right, the character string to the right of the candidate division point may be output, and when a user presses a candidate division point and drags it to the left, the character string to the left of the candidate division point may be output. In this way, when a predetermined operation is performed on a candidate division point, any of the character strings divided at the candidate division point may be output.

[0069] In S508, when the user performs a file name confirmation operation, the CPU 311 of the user terminal 103 confirms the file name to be given to the document image based on the character string input in the file name input field in S506 to S507. Then, the CPU 311 of the user terminal 103 transmits information on the confirmed file name to the image processing server 102, and associates the information on the confirmed file name with the document image.

[0070] In this embodiment, in steps S506 to S508, after the file name is confirmed on the UI screen displayed on the user terminal 103, information on the confirmed file name is displayed on the image processing server 102, but this is not limited to this. For example, each time the user specifies a character string or operates a candidate division point on the user terminal 103, information on the input or changed character string may be notified to the image processing server 102.

[0071] By applying this image processing as described above, the user can select the character recognition result or text segmentation result and assign it as a file name. Furthermore, if the region is not the result desired by the user, the character recognition result or text segmentation result can be corrected by performing a specified operation on the candidate segmentation points, and the output text can be corrected accordingly.

[0072] In this embodiment, the language setting for the character recognition process has been described in Japanese, but the present invention is not limited to this. When the character recognition language is English, the regular expression definition corresponding to English may be read and executed. Furthermore, the language may not be specified by the user, and language may be estimated for each line during character recognition, and the regular expression definition to be read during text segmentation may be changed and executed for each language estimation result. Furthermore, the form may be classified before character recognition, and the regular expression definition to be read during text segmentation may be changed and executed for each classification result.

[0073] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0074] 100 Image Processing System 101 Image forming device 102 Image processing server (image processing device) 103 User terminal 104 Network

Claims

1. a first display control means for controlling display of a UI for setting a file name of a file including a document image obtained by scanning a document, the UI including the document image; a second display control means for controlling to display a frame representing an area identified as an area including characters by an analysis process of the document image, which corresponds to a position on the document image designated by a user, to display a first character string, which is a character string recognized by a character recognition process for the area, in a display area within the UI, and to display an item for receiving a user operation in the frame; a receiving means for receiving a user operation to move the item displayed in the frame; a third display control means for controlling, when the user operation for moving the item is performed, a character string displayed in the display area to be changed from the first character string to a second character string that is determined based on the user operation and is a part of the first character string, and then displayed; having The file name of the file is set using the character string displayed in the display area. A terminal device comprising:

2. the second character string is a character string obtained by a first process of dividing the first character string; 2. The terminal device according to claim 1 .

3. The first processing is processing of dividing the first character string based on a position of a predetermined character in the first character string.

3. The terminal device according to claim 2.

4. the first process is a process of dividing the first character string based on a position obtained by searching for a position of the predetermined character in the first character string using a regular expression definition in which a regular expression for searching the predetermined character is associated with a parameter related to a space character; 4. The terminal device according to claim 3.

5. the predetermined character is a space character; 5. The terminal device according to claim 3 or 4.

6. The predetermined character is a colon.

5. The terminal device according to claim 3 or 4.

7. The predetermined character is a semicolon.

5. The terminal device according to claim 3 or 4.

8. The first process is a process of dividing the first character string between a character included in the first character string and a character adjacent to the character when the distance between the character and the character adjacent to the character is equal to or greater than a predetermined value relative to a height of the character.

3. The terminal device according to claim 2.

9. The terminal device is capable of communicating with a server; The server performs the first process.

9. The terminal device according to claim 2, wherein the first and second inputs are connected to the first and second inputs.

10. the first character string is a character string obtained by a second process of dividing a horizontal line of character strings in the document image, the character string being recognized by a character recognition process on the document image; 10. The terminal device according to claim 1 ,

11. The second process is a process of dividing the horizontal string by using a regular expression definition that associates a regular expression with a parameter related to a space character. The terminal device according to claim 10 .

12. The second process is a process of dividing the horizontal string between a character included in the horizontal string and a character adjacent to the character, when the distance between the character and the character adjacent to the character is equal to or greater than a predetermined value relative to the height of the character. The terminal device according to claim 10 .

13. the second display control means, in response to the user designating the position on the document image, highlights the area corresponding to the designated position and displays the item in an operable manner.

13. The terminal device according to claim 1, wherein the first and second terminals are connected to each other.

14. The terminal device is capable of communicating with a server; The server performs the character recognition process.

14. The terminal device according to claim 1, wherein the first and second inputs are connected to the first and second inputs.

15. a first display control means for controlling display of a UI for setting a file name of a file including a document image obtained by scanning a document, the UI including the document image; a second display control means for controlling to display a frame representing an area identified as an area including characters by an analysis process of the document image, which corresponds to a position on the document image designated by a user, to display a first character string, which is a character string recognized by a character recognition process for the area, in a display area within the UI, and to display an item for receiving a user operation in the frame; a receiving means for receiving a user operation to move the item displayed in the frame; a third display control means for controlling, when the user operation for moving the item is performed, a character string displayed in the display area to be changed from the first character string to a second character string that is determined based on the user operation and is a part of the first character string, and then displayed; having The file name of the file is set using the character string displayed in the display area.

1. An image processing system comprising:

16. a first display control means for controlling display of a UI for setting a file name of a file including a document image obtained by scanning a document, the UI including the document image; a second display control step of controlling to display a frame representing an area identified as an area including characters by an analysis process of the document image, the area corresponding to a position specified by a user on the document image, displaying a first character string, which is a character string recognized by a character recognition process on the area, in a display area within the UI, and displaying an item for receiving a user operation in the frame; a receiving step of receiving a user operation to move the item displayed in the frame; a third display control step of controlling, when the user operation for moving the item is performed, a character string displayed in the display area to be changed from the first character string to a second character string that is a part of the first character string and is determined based on the user operation; and having The file name of the file is set using the character string displayed in the display area.

13. An image processing method comprising:

17. A program for causing a computer to function as each of the means of the terminal device according to any one of claims 1 to 14.

Images