Image processing system, image processing method and storage medium providing attribute information

Abstract

The present invention relates to an image processing system, an image processing method and a storage medium for providing attribute information. In the image processing system disclosed in the present invention, a setting window is displayed, the setting window at least including a preview area for previewing a scanned image and a text field for inputting attribute information about the scanned image, when a mouse is hovered over a character area in the scanned image previewed in the setting window, a control is performed to preliminarily display a character string corresponding to the character area where the mouse is hovering in the text field, and when the character area where the mouse is hovering is clicked with a mouse, a control is performed to fix the character string preliminarily displayed in the text field.

Description

Technical Field

[0001] The present disclosure relates to an image processing system for setting attribute information on a scanned image. Background Art

[0002] Conventional image processing systems scan paper business forms and digitize them, and set attributes such as a file name, etc., to the scanned image of the digitized business form based on the recorded content of the business form, thereby managing the business form. One method of setting such attributes is a method of setting the attributes based on a recognition result obtained by performing character recognition processing on the scanned image of the business form.

[0003] In the system as described above, one conceivable method is to set the characters in the image as attributes in response to a click on a character area in a preview image displayed after character recognition of the business form image. Japanese Patent Laid-Open No. 2001-154779 discloses that, as a processing method for clearly indicating the characters that the user wants to set and preventing erroneous selection in a click operation on a character area, a larger mouse cursor is displayed when a larger character is pointed at, and a smaller mouse cursor is displayed when a smaller character is pointed at. Summary of the invention

[0004] In the image processing system of the present invention, the image processing system includes: a display unit, which displays a setting window, the setting window at least including a preview area for previewing a scanned image and a text field for inputting attribute information about the scanned image; and a control unit, which, when a mouse is hovered over a character area within the scanned image previewed in the setting window, controls to preliminarily display a character string corresponding to the character area where the mouse is hovering in the text field, and when the character area where the mouse is hovering is clicked with a mouse, controls to fix the character string preliminarily displayed in the text field.

[0005] Further features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a diagram illustrating the overall configuration of the present disclosure.

[0007] Figure 2 This is a diagram of the hardware structure of the MFP.

[0008] Figure 3 This is a hardware diagram of the client PC and MFP collaborative service server.

[0009] Figure 4 It is a software structure diagram of the present disclosure.

[0010] Figure 5 This is a sequence diagram illustrating the process flow between the devices.

[0011] Fig. 6A is a diagram illustrating an example of a window displayed on the operation unit of the MFP.

[0012] Figure 6B is a diagram illustrating an example of a window displayed on the operation unit of the MFP.

[0013] Figure 7 This is the flow of the display process of the character area of ​​the scanned image in the present disclosure.

[0014] Fig. 8A and Figure 8B is a diagram illustrating an example of a window on the operation unit of the MFP in the second embodiment. DETAILED DESCRIPTION

[0015] Figure 1 1 is a diagram illustrating the overall configuration of the hardware of the system of the first embodiment. The image processing system includes a multifunction peripheral (MFP) 110 as an image processing apparatus, a client PC 111, an MFP cooperative service server 120, and a cloud storage 130. The MFP 110 and the client PC 111 are connected to the server via a local area network (LAN) 112 in a communicable manner, and the server provides various services on the Internet.

[0016] The MFP 110 is a multifunction machine having multiple functions such as a scanner or a printer, and is an example of an image processing apparatus.

[0017] The client PC 111 is a computer that receives a service requested from the MFP cooperative service server 120 .

[0018] The MFP cooperative service server 120 is a server having a function of storing an image file scanned by the MFP 110 on its own server or transmitting such an image file to a service that can store files, such as another storage service.

[0019] The cloud storage 130 corresponds to a service that can store files or retrieve files on a web browser via the Internet.

[0020] Note that although the image processing system of the present embodiment has a configuration formed of the MFP 110, the client PC 111, the MFP cooperative service server 120, and the cloud storage 130, the configuration is not limited thereto. For example, the MFP 110 may have the roles of both the client PC 111 and the MFP cooperative service server 120. Furthermore, instead of being arranged on the Internet, the MFP cooperative service server 120 may be arranged in a server on the LAN 112. Furthermore, the cloud storage 130 may be replaced with a mail server or the like, and the scanned image is attached to a mail and the mail is sent.

[0021] Figure 2 2 is a hardware configuration diagram of the MFP 110. The MFP 110 is formed by a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223. The control unit 210 is formed by the following units 211 to 219 connected to each other via a system bus 240, and controls the overall operation of the MFP 110. The CPU 211 reads the control program stored in the ROM 212, and executes and controls various functions of the MFP 110, such as reading, printing, or communication. The RAM 213 is used as a temporary storage area such as a main memory, a work area, etc. of the CPU 211. Note that although a single CPU 211 uses a single storage area (RAM 213 or HDD 214) to perform each process illustrated in the flowchart described later in this embodiment, the embodiment is not limited thereto. For example, each process may be performed in cooperation with multiple CPUs or multiple RAMs or HDDs. The HDD 214 is a large-capacity storage unit that stores image data or various programs.

[0022] The operation unit I / F 215 is an interface that connects the operation unit 220 and the control unit 210 to each other. The operation unit 220 is equipped with a touch panel or a keyboard or the like, and accepts user operations, inputs, or instructions.

[0023] The printer I / F 216 is an interface that connects the printer unit 221 and the control unit 210 to each other. Image data for printing is transmitted from the control unit 210 to the printer unit 221 via the printer I / F 216, and is printed on a recording medium. The scanner I / F 217 is an interface that connects the scanner unit 222 and the control unit 210 to each other. The scanner unit 222 reads an original placed on an original glass table (not illustrated) or an automatic original feeder (ADF), generates image data, and inputs the generated image data to the control unit 210 via the scanner I / F 217. For the image data generated by the scanner unit 222, in addition to printing out (copying) from the printer unit 221, the MFP 110 can also perform file transmission or mail transmission. The modem I / F 218 is an interface that connects the modem 223 to the control unit 210. The modem 223 performs fax communication of image data with a fax device on the PSTN. The network I / F 219 is an interface that connects the control unit 210 (MFP 110) to the LAN 112. The MFP 110 transmits image data or information to various services on the Internet or receives various information by using the network I / F 219.

[0024] Figure 3 1 is a hardware configuration diagram of the client PC 111 and the MFP cooperative service server 120. Each of the client PC 111 and the MFP cooperative service server 120 is a so-called information processing device, and is formed of a CPU 311, a ROM 312, a RAM 313, a HDD 314, and a network I / F 315 connected to each other via a system bus 316.

[0025] The CPU 311 reads the control program stored in the ROM 312 to perform various processes, thereby controlling the overall operation. The RAM 313 is used as a temporary storage area such as the main memory, work area, etc. of the CPU 311. The HDD 314 is a large-capacity storage unit that stores image data or various programs. The network I / F 315 is an interface that connects the client PC 111, the MFP collaborative service server 120, and the cloud storage 130 to the network. The client PC 111 is connected to the locally deployed LAN 112, and the MFP collaborative service server 120 and the cloud storage 130 are connected to the Internet via the locally deployed LAN 112. In addition, in response to a processing request from another device (MFP 110, etc.), the MFP collaborative service server 120 and the cloud storage 130 send and receive various information via the network I / F 315.

[0026] Figure 4 is a diagram of the software configuration of the image processing system according to this embodiment.

[0027] The MFP 110 is generally divided into two, namely, a native function unit 410 and an additional function unit 420. Each unit included in the native function unit 410 is typically provided to the MFP 110, and the additional function unit 420 is an application additionally installed in the MFP 110. The additional function unit 420 is an application based on Java (registered trademark), and can easily implement the addition of functions to the MFP 110. Note that other additional applications (not illustrated) can be installed in the MFP 110. The native function unit 410 has a scan execution unit 411 and an image data storage unit 412. The additional function unit 420 has a display control unit 421, a scan instruction unit 422, and a collaborative service request unit 423.

[0028] The display control unit 421 displays a UI window for accepting user operations on the liquid crystal display unit having a touch panel function of the operation unit 220 of the MFP 110. For example, the display control unit 421 displays UI windows such as a window for inputting authentication information for accessing the MFP collaborative service server 120, a window for setting a scan and an operation for starting a scan, a preview window, and the like.

[0029] The scan instruction unit 422 requests the scan execution unit 411 to perform a scan process according to a user instruction and a scan setting input via the UI window.

[0030] The scan execution unit 411 receives a scan request including scan settings from the scan instruction unit 422. The scan execution unit 411 generates scanned image data of the document placed on the document glass table and read by the scanner unit 222 via the scanner I / F 217 according to the scan request. The generated scanned image data is sent to the image data storage unit 412. In addition, the scan execution unit 411 sends a scanned image identifier that uniquely indicates the stored scanned image data to the scan instruction unit 422. The scanned image identifier is a string of numbers, symbols, letters, etc. (not illustrated) for uniquely identifying an image scanned by the MFP 110. The image data storage unit 412 stores the scanned image data received from the scan execution unit 411 in the HDD 214.

[0031] The scan instruction unit 422 acquires scan image data corresponding to the scan image identifier received from the scan execution unit 411 from the image data storage unit 412. The scan instruction unit 422 requests the cooperative service request unit 423 to instruct the MFP cooperative service server 120 to process the acquired scan image data.

[0032] The collaborative service request unit 423 requests the MFP collaborative service server 120 to perform various processes, such as logging in, analyzing scanned images, sending scanned images, etc. The collaborative service request unit 423 communicates with the MFP collaborative service server 120 by using a protocol such as REST or SOAP, but other communication methods may also be used.

[0033] The MFP cooperative service server 120 has a request control unit 431 , an image processing unit 432 , a cloud storage access unit 433 , a data management unit 434 , and a display control unit 435 .

[0034] The request control unit 431 is always in a standby state capable of receiving a request from an external device. In response to receiving a processing request, the request control unit 431 instructs the image processing unit 432, the cloud storage access unit 433, and the data management unit 434 to appropriately perform processing according to the request.

[0035] The image processing unit 432 performs recognition processing or image processing on the image, such as character area analysis, optical character recognition (OCR), similar business form determination (which will be described later), and image processing. Figure 5 ), correction of image rotation or tilt, etc.

[0036] The cloud storage access unit 433 requests the cloud storage 130 to process. Generally, cloud services expose various interfaces for storing files in the cloud storage or obtaining stored files by using protocols such as REST or SOAP. The cloud storage access unit 433 operates the cloud storage 130 by using the exposed cloud storage interface.

[0037] The data management unit 434 holds user information, various setting data, and the like managed by the MFP cooperative service server 120 .

[0038] The display control unit 435 responds to a request from a web browser operated on another terminal (not shown) such as a PC or a mobile phone connected via the Internet, and returns window construction information (HTML, CSS, etc.) required for window display. The user can confirm the user information registered in the MFP cooperative service server 120 or change the settings when scanning via the window displayed in the web browser.

[0039] Note that although Figure 4 An example of a configuration in which the additional function unit 420 is installed in the MFP 110 is illustrated in FIG. 1 , but the present invention is not limited to this configuration, and the function of the additional function unit 420 may be included in the client PC 111 .

[0040] Figure 51 is a sequence diagram illustrating the flow of processing between the various devices when an image scanned by the MFP 110 is processed into a file and the file is sent to the cloud storage. In this example, the communication between the various devices is mainly described. Note that although Figure 5 An operation in which the MFP 110 communicates with the MFP cooperative service server 120 is illustrated, but acquisition of analysis results described later, display of a window, instruction of learning, and the like may be performed by the client PC 111 instead of the MFP 110 .

[0041] The MFP 110 displays a main window on the touch panel of the operation unit 220, in which buttons for realizing various functions to be provided are arranged in a normal state. When an additional application (hereinafter referred to as a "scanning application") for scanning a business form and sending its image file to the cloud storage 130 is installed in the MFP 110, a button for using the function of the application is displayed on the main window of the MFP 110. When the button is pressed, a window for sending the scanned business form to the cloud storage 130 is displayed, and the scanning application is performed. Figure 5 The processing illustrated in the sequence.

[0042] In S501 , the scanning application displays a login window (not illustrated) for inputting authentication information for accessing the MFP cooperative service server 120 .

[0043] In S502 , in response to the user inputting a user ID and the like into the login window, the scanning application requests a login to the MFP cooperative service server 120 .

[0044] In S503, the MFP collaborative service server 120 matches and checks the user name and password included in the login request with the user information managed by the data management unit 434 to determine whether the user name and password are correct, and if correct, returns the access token to the MFP 110. Various requests subsequently issued from the MFP 110 to the MFP collaborative service server 120 are provided together with the access token, and the user to be processed can be identified by the access token. The user authentication method is performed by using a well-known method (authorization using basic authentication, digest authentication, authentication using OAuth, etc.).

[0045] The MFP 110 displays a scan setting window in S504 in response to completion of the login process. The user makes settings related to various scan readings, places a paper business form to be scanned on the original glass or ADF, and presses a "scan start" button. In response, the MFP 110 performs scanning and generates data of a scanned image digitized from the paper business form in S505.

[0046] In subsequent S506 , the MFP 110 transmits a request for analysis of the scanned image together with the image generated by the scanning process to the MFP cooperative service server 120 .

[0047] In S507, the request control unit 431 of the MFP cooperative service server 120 requests image analysis from the image processing unit 432 in response to receiving the analysis request for the scanned image. Then, the request control unit 431 of the MFP cooperative service server 120 returns a "processing ID" which is an identifier uniquely identifying the analysis request for the requested scanned image to the MFP 110 without waiting for the end of the image analysis processing.

[0048] On the other hand, the image processing unit 432 of the MFP cooperative service server 120 that has received the image analysis request performs image analysis processing ( S508 to S510 ).

[0049] In S508 of the image analysis process, the image processing unit 432 analyzes the character region present in the image. This is to analyze the arrangement information and the like on the character region.

[0050] Subsequently, in S509, the image processing unit 432 compares the arrangement information about the image scanned in the past with the arrangement information about the image scanned currently by using the arrangement information about the character area in the business form, and determines whether to scan an image having a similar arrangement of the character area. This process is called similar business form determination. The information about the image scanned in the past used in the determination is stored and accumulated by the process of S517 described later.

[0051] Then, in S510, based on the determination result, character recognition processing is performed on the analyzed character area. Figure 7 Details of the display processing based on the results of the image analysis processing of S508 to S510 are described.

[0052] On the other hand, in S511, the MFP 110 uses the "processing ID" received in the response of S506 to periodically (e.g., approximately every several hundred milliseconds to several milliseconds, etc.) confirm the status of the analysis processing of the scanned image related to the "processing ID" with the MFP collaborative service server 120. Although not illustrated, the processing of S511 is periodically continued until a response of the completion of the image processing of the MFP collaborative service server 120 is obtained (until the timing of S512). In response to receiving the request for processing status confirmation of S511, the request control unit 431 of the MFP collaborative service server 120 confirms the processing status of the "processing ID" and returns the processing status as a response.

[0053] In this response, a character string indicating the current processing status is stored in "Status". For example, when "Status" is "Processing", this indicates that the MFP collaborative service server 120 is processing, and when "Status" is "Complete", this indicates the status of processing completion. Note that when processing fails, other statuses such as "Failed" may be returned. In addition, the response when the processing is completed (when "Status" is "Complete") includes the status and information about the analysis results of the scanned image, scan settings, etc. When the image analysis processing is completed, a notification indicating the completion of the image analysis processing is issued from the image processing unit 432 to the request control unit 431.

[0054] After the image analysis process is completed in the MFP cooperative service server 120 , when the MFP 110 confirms the process status with the MFP cooperative service server 120 in S512 , a response indicating the process status of completion of the image analysis process is returned from the request control unit 431 .

[0055] Subsequently, in S513, the MFP 110 requests the request control unit 431 to obtain the image analysis result, and receives a response related to the analysis result from the request control unit 431. The MFP 110 then obtains the image analysis result information from the URL storing the result information included in the response. When there is a similar business form, attribute information as the image analysis result is also included. The MFP 110 displays the scanned business form list window 600 ( Fig. 6A ).

[0056] In S514, once the user selects a scan image from the scan business form list window 600 displayed on the operation unit 220, the MFP 110 displays a property setting window 610 ( 612 ) for setting property information such as a file name on the operation unit 220. Figure 6B ). Note that although Figure 6B 610 is an example of setting a file name and a value corresponding to a company name as attribute information, but the type of attribute (item name) is not limited thereto. For example, as an attribute, an attribute (item name) such as an address or an amount can be newly added, and an attribute value corresponding thereto can be set. Details of the attribute setting window 610 will be described later.

[0057] After the user sets the attribute information for the scanned business form in the attribute setting window, the user presses the "OK" button. In S515, the MFP 110 sends a request to learn the business form to the MFP cooperative service server 120 by using the information on the character area for setting the attribute as input information.

[0058] In S516 , the request control unit 431 of the MFP cooperative service server 120 that has received the information on the character area for setting the attribute requests the image processing unit 432 to learn the business form.

[0059] In S517, the image processing unit 432 stores the information on the character area of ​​the entire image and the information on the character area received in S516 from the user for setting the attribute as input information for learning.

[0060] Furthermore, in S518, if there is a business form similar to the business form stored in S517, the MFP collaborative service server 120 performs similar business form determination processing again. When the learning is finished, the MFP collaborative service server 120 notifies the MFP 110 of the end of the learning. Here, the learning is an operation of storing the business form and the data of the layout information of the character area of ​​the business form in association with each other.

[0061] Then, in S519, the MFP 110 sends the scanned image and attribute information such as a file name set after processing the scanned image into a file to the MFP cooperative service server 120. In response to receiving the request, the MFP cooperative service server 120 replies to the MFP 110 to indicate that the request has been successfully received. In response to receiving the response to the transmission, the MFP 110 ends the processing and moves back to the display of the scan setting window of S504.

[0062] In S520, the request control unit 431 of the MFP cooperative service server 120 starts the file generation process. Here, the MFP cooperative service server 120 acquires information about the file format to be sent to the cloud storage 130 from the scan settings registered in the MFP cooperative service server 120, and generates a file from the scanned image based on the settings.

[0063] Then, in S521, the request control unit 431 of the MFP cooperative service server 120 sets the attribute information received in S519 and sends the generated file to the cloud storage 130. In response to receiving the file, the cloud storage 130 returns a sending completion response to the MFP cooperative service server 120.

[0064] Fig. 6A and Figure 6B 4 is a diagram illustrating an example of windows displayed by the MFP 110. Display control of these windows is performed by the display control unit 421. Note that such windows can be displayed on the client PC 111 as described above.

[0065] Fig. 6A6 is a diagram illustrating an example of a scanned business form list window 600. This window enables viewing a list of business forms after the scanning and image analysis process is completed and before sending to the cloud storage 130 ( Figure 5 In addition, the window is composed of a scanned business form list 601, a send button 602, an edit button 603 and a delete button 604.

[0066] The scanned business form list 601 is an area that displays a list of business forms for which the scanning and image analysis processing (S505 to S510) is completed. This area consists of fields of a business form name 605, a destination 606, a status 607, and a type 608. The business form name 605 is an identifier that uniquely identifies the business form name. The destination 606 is the name of the cloud storage 130 that is the destination of the file of the business form. The status 607 indicates the result when similar business form determination is performed on the business form, and "not yet learned" or "learned" is displayed. "Not yet learned" means that it is determined that there is no similar business form, and "learned" means that it is determined that there is a similar business form. Type 608 indicates the type of the business form. For example, "estimate" or "invoice" is displayed. In addition, for a business form with a status 607 of "learned", the detailed type of the invoice format corresponding to the business form is displayed, such as "invoice AAA" or "invoice BBB". This is associated with the best similar business form determined by the similar business form determination process.

[0067] The send button 602 is a button for sending a business form to the cloud storage 130. The user can select any business form from the scanned business form list 601 and press the send button 602 to perform transmission to the cloud storage 130 displayed in the destination 606. Note that when the transmission is successfully completed, the business form is deleted from the list.

[0068] The edit button 603 is a button for moving to a property setting window 610 described later. The user can select any business form from the scanned business form list 601 and press the edit button 603, and the window moves to the property setting window 610 for the selected business form ( Figure 6B ), which enables users to edit property settings.

[0069] The delete button 604 is a button for deleting a business form. The user can select any business form from the scanned business form list 601 and press the delete button 604 to delete the selected business form.

[0070] Figure 6B is an example property setting window 610 ( Figure 5Attribute area 611 is an area for displaying attribute information set by a user. In attribute area 611, a text field 618 for inputting attribute information and a cutout image 619 of a character area described later are displayed. When a character area is not selected, attribute area 611 is displayed in a state where the text field is empty and the cutout image is not set. In addition, a character area that can be selected by clicking, etc. can be displayed with a changed background color or with a frame surrounding the selected character area so that the user can recognize the character area.

[0071] The preview area 612 displays the scanned image. When the mouse is hovered over a character area of ​​the image, the character area is displayed with a colored frame line surrounding the character area or with its color changed, so that the user can recognize the character area over which the mouse is hovering. In addition, the character string of the character area is preliminarily displayed in the text field 618. In addition, a cropped image of the character area is preliminarily displayed in the cropped image 619. Whenever the character area over which the mouse is hovering changes, the text field 618 and the cropped image 619 are switched. At this time, in order to inform the user that what is displayed is a preliminary setting, the frame line or the cropped image of the character area is displayed in a display form different from the display form applied when the selection is fixed by clicking on the character area (described later). Reference will be made to Figure 7 The flowchart describes the details. Then, when the character area where the mouse is hovering is clicked with the mouse, the selection of the character area corresponding to the click position is fixed. In order to make the selection fixed and visualized, the character area where the selection is fixed is displayed with a graphic such as a line or frame line, a background color, etc. Note that for each attribute to be set, the frame of the selected character area can be displayed in a different color, or when multiple character areas are selected for a single attribute, the color of each character area can be a different color. In addition, the preview display position can be changed so that the selected character area is located in the center, or the magnification can be changed. In addition, the character string initially displayed in the text field 618 can also be displayed with a changed display color to make the character string fixed and visualized. For example, the character string of the text field can be displayed with gray characters during the initial display, and can be displayed with black characters after being fixed.

[0072] When the character area once selected is clicked again, the selection is canceled, and the character string of the corresponding file name is deleted, returning to the state where the line and color added to the character area are not displayed. In the example, when the character string is in the unselected state, the character area is not displayed on the preview.

[0073] However, in order to indicate to the user which area can be clicked, a display with a color or a frame line may be used to make the character area distinguishable. In addition, the display for distinguishing the character area may be switched between a display state and a non-display state by a button, etc. In addition, when a sliding operation is performed on the preview area, the position of the image displayed in the preview area is moved.

[0074] The character area delete button 613 is used to delete the character area linked to the character area delete button 613 and update the attribute area 611 related to the character area. Specifically, the corresponding character string of the text field 618 is deleted, and the cropped image 619 is set to a non-set state.

[0075] The preview zoom-in button 614 increases the magnification of the image displayed in the preview area. The preview zoom-out button 615 decreases the magnification of the image displayed in the preview area. Note that the display position is adjusted so that the center coordinates of the preview area after zooming in or out are the same as the center coordinates of the preview area before zooming in or out. In the case where the display position of the preview image is moved by a slide operation or the display magnification is changed by pressing the preview zoom-in button or the preview zoom-out button, the preview initial display button 616 is used to restore the display magnification and display position to the initial state.

[0076] The OK button 617 is used to send the scanned image together with the file name set by the property setting window 610 to the MFP cooperative service server 120 and perform machine learning processing (S515 to S518). In response to the sending completion, the window returns to the scanned business form list window 600.

[0077] Figure 7 1 is a flowchart illustrating the details of the display processing in this system when the mouse is hovered over the character area. This processing is repeated while the cursor is located on the preview window.

[0078] First, in S1, determine whether the position where the mouse is hovering is in the character area. If it is determined in S1 that the position where the mouse is hovering is in the character area (yes), the process proceeds to S2, and if it is determined in S1 that the position where the mouse is hovering is not in the character area (no), the process proceeds to S7.

[0079] If the mouse hovers in the character area, then in S2, the character area where the mouse is hovering is displayed so that the user can recognize it. For example, the character area is displayed with graphics such as lines or frame lines, or with a changed background color.

[0080] In subsequent S3, the image is cropped along the frame of the character region where the mouse is hovering, thereby extracting the cropped image.

[0081] In S4, the character string in the character area where the mouse is hovering is extracted. At S4, OCR processing can be performed, and the character string can be extracted. In addition, OCR processing can be performed on the entire character area in the preview image in advance, or the result of the character recognition processing performed on the entire character area in S510 can be obtained and used as a response to the request to obtain the analysis result of S513.

[0082] In S5, the cropped image 619 extracted in S3 is displayed in the attribute area. Furthermore, in S6, the character string extracted in S4 is displayed in the text field 618 in the attribute area, and the flow ends.

[0083] Note that if it is determined in S1 that the mouse hover position is not in the character area, then in S7, the frame or line added to the character area is removed, the cropped image and text field of the attribute area are restored to the original display, and the process ends.

[0084] Next, the second embodiment will be described. In the first embodiment, when Figure 7 When the mouse is hovered over the character area in the determination process, the cropped image and the character string of the character area are displayed in the attribute area, and this enables the user to confirm whether the correct character area is set. However, since a graphic or color such as a frame or line is added to the character area in a recognizable manner only when the mouse is hovered over the character area, the user is unlikely to be aware of the mouse hovering operation on the character area. In the second embodiment, an example of adding a graphic or color such as a frame, line, etc. to the entire character area within the preview image when the mouse is hovered over the preview area displaying the preview image will be described.

[0085] Fig. 8A and Figure 8B is a diagram illustrating an example of the property setting window 610 .

[0086] Fig. 8A There is illustrated a case where the mouse cursor 801 is outside the area of ​​the preview image 802. At this point in time, neither a graphic such as a frame or a line nor a color is added to the character area within the preview image 802.

[0087] On the other hand, when the mouse cursor 801 is within the area of ​​the preview image, as shown in FIG. Figure 8B As shown, a graphic or color such as a frame or line is added so that the entire character area (803) within the preview image 802 can be identified. Then, in this state, as described in the first embodiment, when the mouse is hovered over the character area within the preview image, a color or graphic different from the color or graphic added to the entire character area is added to the character area where the mouse is hovering, thereby displaying the character area in a recognizable manner.

[0088] Other embodiments

[0089] The present invention has been described as a system combining an image processing device (MFP), an MFP cooperative service server, and cloud storage, but the functions described may be provided separately in other devices. For example, all functions may be provided in the image processing device (MFP), or the service server may be a server with storage functions.

[0090] Furthermore, in the present invention, although a similar business form is determined based on arrangement information of character regions on a business form, determination may be made using a neural network that has learned the relationship between arrangement information on character regions and the business form.

[0091] Furthermore, the function of selecting a character area of ​​an image in the present invention is applicable to any information processing apparatus that can display an image and can select an image.

[0092] The embodiments of the present disclosure may also be implemented by reading and executing computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more completely referred to as a "non-transitory computer-readable storage medium") to perform one or more functions in the above-mentioned embodiments, and / or a computer of a system or device including one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing one or more functions in the above-mentioned embodiments, and the embodiments of the present invention may be implemented by, for example, reading and executing the computer executable instructions from the storage medium by the computer of the system or device to perform one or more functions in the above-mentioned embodiments, and / or controlling the one or more circuits to perform one or more functions in the above-mentioned embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a memory of a distributed computing system, an optical disk (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc (BD) TM ), flash memory devices, memory cards, etc.

[0093] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0094] While the present disclosure has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image processing system, comprising: a display unit that displays a setting window, the setting window including at least a preview area for previewing the scanned image and a text field for inputting attribute information about the scanned image; as well as A control unit that, when a mouse is hovered over a character area within a scanned image previewed in a setting window, controls to preliminarily display a character string corresponding to the character area where the mouse is hovering in a text field and preliminarily displays a cropped image of the character area where the mouse is hovering, and that, when the character area where the mouse is hovering is clicked with a mouse, controls to fix the character string preliminarily displayed in the text field.

2. The image processing system according to claim 1, wherein: When the mouse is hovered over a character area in the scanned image previewed in the setting window, the control unit also changes the display of the character area where the mouse is hovered so that the character area where the mouse is hovered can be identified in the scanned image displayed in the preview area.

3. The image processing system according to claim 1, wherein: When the mouse is hovered over a character area in the scanned image previewed in the setting window, the control unit further displays a frame surrounding the character area where the mouse is hovered so that the character area where the mouse is hovered can be identified in the scanned image displayed in the preview area.

4. The image processing system according to claim 1, wherein: When a mouse cursor enters the preview area, the control unit displays the entire character area in the preview image so that the user can recognize it, and, when the mouse is hovered over the character area in the previewed scanned image, further controls to change the display of the character area where the mouse is hovering in a form different from the form of display for recognizing the entire character area, so that the character area where the mouse is hovering can be recognized in the scanned image displayed in the preview area. 5 . The image processing system according to claim 1 , further comprising a region analysis unit configured to identify a character region within the scanned image by performing region analysis on the scanned image. 6 . The image processing system according to claim 1 , further comprising a character recognition unit, which acquires a character string corresponding to the character area by performing character recognition processing on the character area in the scanned image. 7 . The image processing system according to claim 1 , further comprising a storage unit that stores the character string fixed by the control unit as attribute information on the scanned image.

8. The image processing system according to claim 7, wherein: The storage unit stores the character string fixed by the control unit in an external memory as attribute information on the scanned image.

9. An image processing method, comprising: displaying a setting window, the setting window comprising at least a preview area for previewing the scanned image and a text field for inputting attribute information about the scanned image; as well as When the mouse is hovered over a character area within a scanned image previewed in a setting window, control is performed to preliminarily display a character string corresponding to the character area where the mouse is hovering in a text field and to preliminarily display a cropped image of the character area where the mouse is hovering, and when the character area where the mouse is hovering is clicked with a mouse, control is performed to fix the character string preliminarily displayed in the text field.

10. A computer-readable storage medium storing a program, wherein: The program causes the computer to: displaying a setting window, the setting window comprising at least a preview area for previewing the scanned image and a text field for inputting attribute information about the scanned image; as well as When the mouse is hovered over a character area within a scanned image previewed in a setting window, control is performed to preliminarily display a character string corresponding to the character area where the mouse is hovering in a text field and to preliminarily display a cropped image of the character area where the mouse is hovering, and when the character area where the mouse is hovering is clicked with a mouse, control is performed to fix the character string preliminarily displayed in the text field.

Images