Information processing apparatus, control method, program, and system

Abstract

To easily make a transition to a preview state in which a user can easily perform a touch operation in a character area in a scanned image.SOLUTION: An image processing apparatus according to the present disclosure comprises: detection means that detects a character area from a scanned image obtained by scanning a document; and display control means that displays a UI screen for setting properties to the scanned image. A preview area for displaying the scanned image is present on the UI screen. The display control means determines a predetermined magnification variation ratio on the basis of the character area detected by the detection means, stores the determined predetermined magnification variation ratio, performs initial display in the preview area by using the scanned image enlarged or reduced on the basis of the predetermined magnification variation ratio, and after the preview area is changed from a state of the initial display, according to detection of an operation of a predetermined button present on the UI screen, returns the preview area to the state of the initial display on the basis of the stored magnification variation ratio.SELECTED DRAWING: Figure 19

Description

Technical Field

[0001] The present invention relates to an image processing technique for preview display of scanned images.

Background Art

[0002] Conventionally, as a method for managing documents, a method of converting a scanned image obtained by reading a document with a scanner into a file in a predetermined format and transmitting it to a storage server on a network for storage has been widely used. And by performing preview display of the scanned image before file transmission, the user can check the scanned state in advance. Patent Document 1 discloses a technique for changing the magnification or reduction rate during preview display by means of magnification / reduction buttons.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As a method for easily setting a file name or the like for a scanned image, there is a method in which the user touches and selects a character area on the scanned image being previewed. Generally, since the size of the operation touch panel provided in OA devices such as MFPs is often small, when setting a file name or the like by adopting the above method, it is necessary to expand and contract the scanned image for preview display so that the user can easily select the character area. In this regard, in the technique of Patent Document 1 in which the user performs a button operation to expand and contract the preview display part, it takes time for the user to obtain the desired preview state.

[0005] The present invention is made to address the above problems. That is, the purpose is to enable the user to easily transition to a preview state where it is easy to touch-operate the character area in the scanned image.

Means for Solving the Problems

[0006] An image processing apparatus according to the present disclosure includes a detection unit that detects a character area from a scanned image obtained by scanning a document, and a display control unit that displays a UI screen for setting properties for the scanned image. The UI screen has a preview area for displaying the scanned image. The display control unit determines a predetermined magnification based on the character area detected by the detection unit, stores the determined predetermined magnification, and performs an initial display in the preview area using the scanned image scaled based on the predetermined magnification. After the preview area changes from the initial display state, in response to detecting an operation of a predetermined button present on the UI screen, the preview area is returned to the initial display state based on the stored magnification. This is the gist of the present invention.

Effects of the Invention

[0007] According to the technology of the present disclosure, the user can easily transition to a preview state where it is easy to touch-operate the character area in the scanned image. Thereby, the user's labor can be saved.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential for the solution means of the invention.

[0010] [Embodiment 1] <System Configuration> FIG. 1 is a diagram showing the overall configuration of an image processing system according to the present embodiment. The image processing system includes an MFP (Multifunction Peripheral) 110, and server devices 120 and 130 that provide cloud services on the Internet. The MFP 110 is communicably connected to the server devices 120 and 130 via the Internet.

[0011] The MFP 110 is an example of an information processing device having a scan function. The MFP 110 is a multifunction device having a plurality of functions such as a print function and a BOX storage function in addition to the scan function. The server devices 120 and 130 are both examples of information processing devices that provide cloud services. The server device 120 of the present embodiment provides a cloud service that performs image analysis on a scanned image received from the MFP 110 or transfers a request from the MFP 110 to the server device 130 that provides another service. Hereinafter, the cloud service provided by the server device 120 will be referred to as the "MFP cooperation service". The server device 130 provides a cloud service (hereinafter referred to as the "storage service") that stores a file sent via the Internet or provides a stored file in response to a request from a web browser such as a mobile terminal (not shown). In the present embodiment, the server device 120 that provides the MFP cooperation service will be referred to as the "MFP cooperation server", and the server device 130 that provides the storage service will be referred to as the "storage server".

[0012] The configuration of the image processing system 100 shown in FIG. 1 is an example and is not limited thereto. For example, the MFP 110 may also have the functions of the MFP cooperation server 120. Further, the MFP cooperation server 120 may be connected to the MFP 110 via a LAN (Local Area Network) instead of over the Internet. Further, the storage server 130 may be replaced with a mail server that performs a mail delivery service and applied to a scenario where a scanned image of a document is attached to and sent by mail.

[0013] <Hardware Configuration of MFP> FIG. 2 is a block diagram showing the hardware configuration of the MFP 110. The MFP 110 includes a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223. The control unit 210 includes the following units 211 to 219 and controls the overall operation of the MFP 110. The CPU 211 reads and executes various control programs (programs corresponding to various functions shown in the software configuration diagram described later) stored in the ROM 212. The RAM 213 is used as a temporary storage area such as the main memory and work area of the CPU 211. In this embodiment, it is assumed that one CPU 211 executes each process shown in the flowchart described later using one memory (RAM 213 or HDD 214), but it is not limited to this. For example, multiple CPUs, multiple RAMs, or HDDs may cooperate to execute each process. The HDD 214 is a large-capacity storage unit that stores image data and various programs. The operation unit I / F 215 is an interface that connects the operation unit 220 and the control unit 210. The operation unit 220 is equipped with a touch panel, a keyboard, etc., and receives operations / inputs / directions from the user. Note that the touch operation on the touch panel includes operations by a human finger and operations by a touch pen. The printer I / F 216 is an interface that connects the printer unit 221 and the control unit 210. Print image data is transferred from the control unit 210 to the printer unit 221 via the printer I / F 216 and printed on a recording medium such as paper. The scanner I / F 217 is an interface that connects the scanner unit 222 and the control unit 210. The scanner unit 222 optically reads a document set on a document table or ADF (Auto Document Feeder) (not shown) to generate scan image data and inputs it to the control unit 210 via the scanner I / F 217. The scan image data generated by the scanner unit 222 can be printed by the printer unit 221 (copy output), saved in the HDD 214, or file-transmitted to an external device such as the MFP cooperation server 120 via the LAN. The modem I / F 218 is an interface that connects the modem 223 and the control unit 210. The modem 223 performs facsimile communication of image data with a facsimile device (not shown) on the PSTN.The network I / F 219 is an interface that connects the control unit 210 (MFP 110) to the LAN. The MFP 110 uses the network I / F 219 to transmit scanned image data to the MFP cooperation server 120 or receive various data from the MFP cooperation server 120. The hardware configuration of the MFP 110 described above is an example, and it may be provided with other configurations as necessary, or may not have some configurations.

[0014] <Hardware Configuration of Server Device> FIG. 3 is a block diagram showing the hardware configuration of the MFP cooperation server 120 / storage server 130. The MFP cooperation server 120 and the storage server 130 have a common hardware configuration and are composed of a CPU 311, a ROM 312, a RAM 313, an HDD 314, and a network I / F 315. The CPU 311 controls the overall operation by reading out the control program stored in the ROM 312 and executing various processes. The RAM 313 is used as a temporary storage area such as the main memory and work area of the CPU 311. The HDD 314 is a large-capacity storage unit that stores image data and various programs. The network I / F 315 is an interface that connects the control unit 310 to the Internet. The MFP cooperation server 120 and the storage server 130 receive requests for various processes from other devices (such as the MFP 110) via the network I / F 315 and return the processing results corresponding to the requests.

[0015] <Software Configuration of Image Processing System> FIG. 4 is a block diagram showing the software configuration of the image processing system 100 according to the present embodiment. Hereinafter, the software configurations corresponding to the roles of the MFP 110, the MFP cooperation server 120, and the storage server 130 that constitute the image processing system 100 will be described in order. In the following, among the various functions of each device, the description will be limited to the functions related to the process of scanning a document, creating a file, and storing it in the storage server 130.

[0016] <<Software Configuration of MFP>> The functional modules of the MFP 110 are roughly classified into two types: a native functional module 410 and an additional functional module 420. The native functional module 410 is an application that is standardly provided in the MFP 110, while the additional functional module 420 is an application that is additionally installed in the MFP 110. The additional functional module 420 is a Java (registered trademark)-based application, which can easily realize the addition of functions to the MFP 110. Note that other additional applications (not shown) may be installed in the MFP 110.

[0017] The native functional module 410 has a scan execution unit 411 and a scanned image management unit 412. The additional functional module 420 has a display control unit 421, a scan control unit 422, a cooperation service request unit 423, and an image processing unit 424.

[0018] The display control unit 421 displays a user interface screen (UI screen) for receiving various user operations on the touch panel of the operation unit 220. The various user operations include, for example, input of login authentication information for accessing the MFP cooperation server 120, scan settings, scan start instructions, file name settings, file save instructions, and the like.

[0019] The scan control unit 422 instructs the scan execution unit 411 to execute a scan process along with scan setting information in response to a user operation made on the UI screen (for example, pressing the "Scan Start" button). The scan execution unit 411 causes the scanner unit 240 to perform a document reading operation via the scanner I / F 217 according to the scan process execution instruction from the scan control unit 422, and generates scan image data. The generated scan image data is stored in the HDD 214 by the scan image management unit 412. At this time, information on a scan image identifier that uniquely indicates the stored scan image data is notified to the scan control unit 422. The scan image identifier is a number, symbol, alphabet, etc. for uniquely identifying an image scanned in the MFP 110. The scan control unit 422 acquires, for example, scan image data to be filed from the scan image management unit 412 using the above scan image identifier. Then, it instructs the cooperation service request unit 423 to make a request to the MFP cooperation server 120 for the processes necessary for filing.

[0020] The cooperation service request unit 423 makes requests for various processes to the MFP cooperation server 120 and receives responses therefrom. The various processes include, for example, login authentication, analysis of scan images, transmission of scan image data, etc. Communication protocols such as REST or SOAP are used for the interaction with the MFP cooperation server 120.

[0021] The image processing unit 424 performs predetermined image processing on the scan image data to generate an image used on the UI screen displayed by the display control unit 421. Details of the predetermined image processing will be described later.

[0022] <<Software Configuration of Server Device>> First, the software configuration of the MFP cooperation server 120 will be described. The MFP cooperation server 120 includes a request control unit 431, an image processing unit 432, a storage server access unit 433, a data management unit 434, and a display control unit 435. The request control unit 431 waits in a state where it can receive requests from external devices, and according to the received request content, instructs the image processing unit 432, the storage server access unit 433, and the data management unit 434 to execute predetermined processes. The image processing unit 432 performs image analysis processes such as detection processing of character areas, character recognition processing, and determination processing of similar documents on the scanned image data sent from the MFP 110, and also performs image processing such as rotation and inclination correction. The storage server access unit 433 makes requests for processes to the storage server 130. In cloud services, various interfaces for saving files to the storage server and retrieving saved files using protocols such as REST and SOAP are provided. The storage server access unit 433 makes requests to the storage server 130 using the provided interfaces. The data management unit 434 holds and manages user information, various setting data, etc. managed by the MFP cooperation server 120. The display control unit 435 receives requests from web browsers operating on PCs and mobile terminals (both not shown) connected via the Internet, and returns screen configuration information (HTML, CSS, etc.) necessary for screen display. The user can check the registered user information and change the scan settings via the screen displayed on the web browser.

[0023] Next, the software configuration of the storage server 130 will be described. The storage server 130 includes a request control unit 441, a file management unit 442, and a display control unit 443. The request control unit 441 waits in a state where it can receive requests from external devices. In this embodiment, in response to a request from the MFP cooperation server 120, it instructs the file management unit 442 to save the received file or read the saved file. Then, it returns a response according to the request to the MFP cooperation server 120. The display control unit 443 receives a request from a web browser operating on a PC or a mobile terminal (both not shown) connected via the Internet and returns screen configuration information (such as HTML, CSS) necessary for screen display. The user can check or acquire the saved file via the screen displayed on the web browser.

[0024] <Overall processing flow of the image processing system> FIG. 5 is a sequence diagram showing the processing flow between devices when a document is scanned by the MFP 110, the obtained scanned image is filed, and saved in the storage server. FIG. 6 is a diagram showing an example of the UI screen of the main menu (hereinafter referred to as the "main screen") displayed when the MFP 110 is started. By scanning a document, filing it, and installing a dedicated application necessary for using the cloud storage service in the MFP 110, a button 601 "Scan and save to cloud storage" is displayed on the main screen 600. Then, when the user presses the button 601 "Scan and save to cloud storage" from the menu buttons displayed within the main screen 600, a series of processes shown in the sequence diagram of FIG. 5 start. Hereinafter, the interaction between devices will be described in chronological order according to the sequence diagram of FIG. 5. In the following description, the symbol "S" represents a step.

[0025] The MFP 110 displays a UI screen (hereinafter referred to as the "login screen") for inputting login authentication information for accessing the MFP cooperation server 120 on the operation unit 220 (S501). An example of the login screen is shown in FIG. 7. When the user inputs a pre-registered user ID and password into the input fields 702 and 703 on the login screen 700 and presses the "Login" button 701, a login authentication request is sent to the MFP cooperation server 120 (S502).

[0026] Upon receiving the login authentication request, the MFP cooperation server 120 performs authentication processing using the user ID and password included in the request (S503). If it is confirmed as a legitimate user as a result of the authentication processing, the MFP cooperation server 120 returns an access token to the MFP 110. Thereafter, by sending this access token together in various requests made from the MFP 110 to the MFP cooperation server 120, the logged-in user is identified. In this embodiment, it is assumed that upon completion of logging in to the MFP cooperation server 120, logging in to the storage server 130 is also completed simultaneously. For this purpose, the user shall previously associate the user ID for using the MFP cooperation service with the user ID for using the storage service via a web browser or the like of a PC (not shown) on the Internet. Thereby, if the login authentication to the MFP cooperation server 120 is successful, the login authentication to the storage server 130 is also completed simultaneously, and the operation for logging in to the storage server 130 can be omitted. And in the MFP cooperation server 120, it becomes possible to respond to requests regarding the storage service from the user who has logged in to the own device. Note that the login authentication method may be performed using a generally known method (such as Basic authentication, Digest authentication, authorization using OAuth).

[0027] When the login is completed, the MFP 110 displays a UI screen for scan settings (hereinafter referred to as the "scan settings screen") on the operation unit 220 (S507). An example of the scan settings screen is shown in FIG. 8. On the scan settings screen 800, there are a "scan start" button 801, a color setting column 802, and a resolution setting column 803. The "scan start" button 801 is a button for instructing the start of scan processing for a document set on the document table (assuming documents such as estimates and invoices in this embodiment). In the color setting column 802, the color mode at the time of scanning is set. For example, it can be specified from options such as full color and monochrome. In the resolution setting column 803, the resolution at the time of scanning is set. For example, it can be specified from options such as 600 dpi and 1200 dpi. Note that the color mode and resolution are examples of setting items, and not all of them need to exist, or other setting items may exist. Also, the options regarding the color mode and resolution may be limited to only the setting values required by the storage service. The logged-in user makes detailed condition settings for the scan processing via such a scan settings screen 800. When the logged-in user who has completed the scan settings sets the document to be scanned on the document table of the MFP 110 and presses the "scan start" button 801, the scan is executed (S505). As a result, image data obtained by digitizing the paper document is generated. After the completion of the scan, the MFP 110 transmits the image data obtained by the scan to the MFP cooperation server 120 together with its analysis request (S506).

[0028] In the MFP cooperation server 120 that has received the analysis request for the scan image, the request control unit 431 instructs the image processing unit 432 to execute the image analysis process (S507). At that time, the request control unit 431 returns a request ID that can uniquely identify the received analysis request to the MFP 110. An example of the request ID is shown in FIG. 9(a). On the other hand, the image processing unit 432 that has received the instruction to execute the analysis process executes the analysis process on the scan image (S508). In this image analysis process, first, a process of detecting character regions existing in the scan image is performed. For the detection of character regions, known methods such as a method of extracting a rectangular region presumed to be a character from a binarized image at a certain threshold value may be applied. Next, for each character region found by the character region detection process, an optical character recognition (OCR) process is performed. Then, a process of determining whether the form to be scanned this time is similar to the forms scanned in the past (similar document determination process) is performed. In this determination of similar forms, arrangement information indicating where each character region existing in the scan image is located in the scan image is used. Specifically, the arrangement information for the past scan image and the arrangement information for the current scan image are compared to determine whether the arrangements of the character regions are similar. This is based on the fact that if the arrangements of the character regions are similar, it can be inferred that they are forms created using the same format. The arrangement information of the past scan image used in this determination of similar forms is accumulated by the "form learning process" described later. The results obtained by these image analysis processes are passed to the request control unit 431. Note that in this embodiment, the similarity or dissimilarity of the forms is determined based only on the degree of similarity of the arrangements of the character regions. However, for example, the type of the form (estimate, invoice, etc.) may be specified from the OCR result, and the similarity or dissimilarity may be determined taking into account the obtained type information.

[0029] While the above-described image analysis process is being performed, the MFP 110 periodically (for example, every few hundred milliseconds to a few milliseconds) makes an inquiry about the processing status to the MFP cooperation server 120 using the above-described request ID (S509 to S509'). This inquiry is repeatedly executed until an image analysis process completion response (S510) from the MFP cooperation server 120 can be obtained. When receiving an inquiry about the processing status, the MFP cooperation server 120 checks the progress of the image analysis process corresponding to the request ID, and if it is not completed, returns a response indicating that it is in progress (see Fig. 9(b)). Also, if it is completed, a response indicating completion (see Fig. 9(c)) is returned. In the "status" of this response, a character string indicating the current processing status is entered. Specifically, when the processing is being performed in the MFP cooperation server 120, "processing" is entered, and when the processing is completed, "completed" is entered. Note that a character string representing another status, such as "failed" when the processing fails, may also be entered. As shown in Fig. 9(c), the response at the time of processing completion includes, in addition to the status information, information regarding the analysis result of the scanned image, etc.

[0030] After receiving the processing completion response, the MFP 110 requests the MFP cooperation server 120 for the result of the image analysis process using the URL included in the response and indicating the storage destination of the image analysis result (S511). In this case, the URLs are "ocrResultUrl" and "matchingResultUrl". Upon receiving the request, the request control unit 431 of the MFP cooperation server 120 returns the result information of the image analysis process.

[0031] Then, the MFP 110 uses the acquired result information to display a UI screen for setting a file name (hereinafter referred to as "file name setting screen") (S512). FIG. 10 shows an example of the file name setting screen. When the user sets a file name and presses the send button 1007, the MFP 110 first sends a learning request including information on the input operation performed by the user when setting the file name (input information at the time of setting) to the MFP cooperation server 120 (S513). FIG. 11 shows an example of the learning request. In "learningContent", the learning content is specified, which includes "rectInfoArray" related to the character area used in the file name and "metadataArray" related to the metadata. "rectInfoArray" contains the coordinate information of the character area used when setting the file name. "metadataArray" contains information on the character area corresponding to the character string used in the file name and information on the delimiter character (separator) attached to the character string. The example in FIG. 11 shows a file name configuration in which the character string of the character area starting with "fileRegion0" is followed by the delimiter character and then the character string of the character area ending with "fileRegion1".

[0032] In the MFP cooperation server 120 that has received the learning request, the request control unit 431 instructs the image processing unit 432 to execute the form learning process (S514). In response to this instruction, the image processing unit 432 saves the arrangement information of each character area existing in the scanned image and the input information (file name configuration information) at the time of file name setting included in the learning request received in S513 (S515).

[0033] After that, the MFP 110 transmits a save request for the scanned image to the MFP cooperation server 120 together with the data of the scanned image and information such as the file name set at the time of file creation (S516). In the MFP cooperation server 120 that has received the save request, the request control unit 431 starts the file generation process and returns a response indicating that the save request has been received normally to the MFP 110. The MFP 110 that has received the response ends the process and returns to the display state of the scan setting screen (S504). On the other hand, in the MFP cooperation server 120, information on the file format is acquired from the previously registered scan settings, and the scanned image is filed according to the file format (S517). At this time, the generated file will be attached with the file name specified in the save request. The scanned image file thus generated is transmitted to the storage server 130 and stored (S518).

[0034] The above is the processing flow of the entire image processing system.

[0035] <Details of MFP Processing> Subsequently, focusing on the operation of the MFP 110, the processing when using the storage service in the above-described image processing system will be described in detail. FIG. 12 is a flowchart showing the processing flow in the MFP 110. This series of processes is realized by the CPU 211 in the control unit 210 executing the control program stored in the HDD 214, and is started in response to the pressing of the "Scan and Save to Cloud Storage" button 601 on the main screen 600 described above. Hereinafter, a detailed description will be given along the flowchart shown in FIG. 12. Note that the symbol "S" at the beginning of each process means a step.

[0036] In S1201, the display control unit 421 displays the above-described login screen 700. When a user name and a password are input to the input fields 702 and 703 on the login screen 700 respectively and the pressing of the "Login" button 701 is detected, the process proceeds to S1202.

[0037] In S1202, the cooperation service request section 423 transmits a login authentication request to the MFP cooperation server 120. When it is confirmed that the user is a legitimate user through login authentication on the MFP cooperation server 120, the cooperation service request section 423 receives an access token from the MFP cooperation server 120.

[0038] In S1203, on the MFP 110 that has received the access token, the display control section 421 displays the above-described scan setting screen 800 on the operation section 220. When a document is set on the platen by the logged-in user and it is detected that the "Scan Start" button 801 is pressed, the display control section 421 notifies the scan control section 422 to that effect.

[0039] In S1204, the scan control section 422 that has received the above notification instructs the scan execution section 411 to execute a scan process. Upon receiving the instruction, the scan execution section 411 scans the document placed on the platen. In the description of this flow, it is assumed that a form (more specifically, a quotation) is used as an example of the scan target. Thereby, a scan image of the quotation as shown in FIG. 13 is obtained. The scan image data generated by the scan is stored in the image data storage section 412, and an identifier capable of identifying the scan image is notified to the scan control section 421. At this time, a message screen (not shown) indicating that the scan is in progress may be displayed on the operation section 220.

[0040] Next, in S1205, the cooperation service request section 423 acquires the scan image data via the scan control section 421 and transmits it to the MFP cooperation server 120 together with an analysis request therefor. Based on the analysis request, the MFP cooperation server 120 executes the above-described image analysis process on the received scan image data. At that time, a request ID is returned from the request control section 431 of the MFP cooperation server 120.

[0041] In S1206, the cooperation service request unit 423 that has obtained the request ID sends a request for acquiring the processing status to the MFP cooperation server 120 together with the request ID. Based on the acquisition request, the MFP cooperation server 120 returns a response corresponding to the progress of the image analysis process specified by the request ID to the MFP110.

[0042] In S1207, if the status in the received response indicates "processing" which means the process is in progress, the cooperation service request unit 423 returns to S1206. Then, after waiting for a predetermined time, it sends a request for acquiring the processing status to the MFP cooperation server 120 again. If the status in the received response indicates "completed" which means the process is completed, it proceeds to S1208. Here, the information on the image analysis result included in the response at the completion of the process in this embodiment will be described with reference to FIG. 9(c) mentioned above. The information on the image analysis result includes three types of information: "ocrResultUrl", "matchingResultUrl", and "formKeys". "ocrResultUrl" is information on the URL for acquiring the result of the OCR process in the image analysis process. "matchingResultUrl" is information on the URL for acquiring the result of the similar form determination process in the image analysis process. "formKeys" is information on the setting values for setting properties such as file name, metadata, and folder path to the scanned image, and is pre-set in the MFP cooperation server 120. The following information is included in "formKeys". · key: A value that uniquely indicates the setting value to be set to the scanned image · keyType: A value indicating the type of the setting value of the key · value: The initial value of the setting value (for example, "scan" indicating that it is a scanned image) · type: A value indicating the type of the value to be input to the setting value (For example, if it is a string, it is "string", if it is a numerical value, it is "number", if it is a date, it is "date", etc.) ·displayName: The display name when the setting screen is displayed on the touch panel of the MFP ·required: A value indicating whether it is mandatory to enter the setting value ·multiSelect: A value indicating whether to use a string of multiple character regions for the file name ·separator: The content of the delimiter character that connects between strings when using a string of multiple character regions ·autoInput: A value indicating whether to perform automatic input of the initial file name

[0043] Figure 9(c) shows "formKeys" when setting a file name for a scanned image. Taking the case of this Figure 9(c) as an example, it will be explained how the values of "value", "displayName", "required", and "multiSelect" affect the file name setting screen (Figure 10) displayed in the following S1209. Now, the content of "value" is empty. In this case, no initial value will be displayed in the file name input field 1001 of the file name setting screen 1000. Also, "displayName" is "File Name". In this case, a string "File Name" indicating the setting title or label, like in the file name setting screen 1000 of Figure 10, will be displayed at the top of the screen (in white characters in this example). Also, "required" is "true". In this case, the send button 1007 cannot be pressed when the file name input field 1001 is in an empty state without a string entered. Also, "multiSelect" is "true". In this case, it becomes possible to select multiple character regions detected from the scanned image, and thus a file name can be set by connecting the strings corresponding to each character region. The above content included in these "formKeys" is set by the user via the file name condition setting screen displayed by the display control unit 435 of the MFP cooperation server 120. An example of the file name condition setting screen is shown in Figure 14. The checkbox 1401 in the file name condition setting screen 1400 is used for the user to select whether to automatically input the file name using the information of the character regions used when setting the file name of the scanned image previously. In the setting column 1402, when using multiple strings corresponding to multiple character regions for the file name, a delimiter character to be inserted between the strings is set from the options (underscore, hyphen, etc.) displayed in a pull-down menu. Here, the underscore is selected. In the setting column 1403, a string (initial file name) to be pre-entered in the input state at the initial display of the file name setting screen (Figure 10) and presented to the user as a candidate for the file name is set. The content set in this file name condition setting screen 1400 is held by the data management unit 434 and is referred to when creating a response at the end of the process.

[0044] In S1208, the cooperation service request unit 423 accesses the URL included in the response at the completion of processing to obtain the image analysis result. In the example of Fig. 9(c), since information is stored in "ocrResultUrl" and "matchingResultUrl", access these two URLs to obtain the image analysis result. The following Table 1 shows the coordinates of the character regions detected from the scan image in Fig. 13 and the character recognition results (recognized character strings) for each character region.

[0045]

Table 1

[0046] And Fig. 15 shows the character recognition result for the scan image in Fig. 13 obtained by accessing "ocrResultUrl". "imageWidth" indicates the number of pixels in the X direction (horizontal direction) of the scan image to be analyzed. "imageHeight" indicates the number of pixels in the Y direction (vertical direction) of the scan image to be analyzed. "regions" includes the coordinate information ("rect") of the character regions detected from the scan image and the character string information ("Text") corresponding to the character regions. "rect" contains information for specifying the detected character region, "x" is the X coordinate of the upper left corner of the character region, "y" is the Y coordinate of the upper left corner of the character region, "width" is the number of pixels in the X (width) direction of the character region, and "height" is the number of pixels in the Y (height) direction of the character region. "text" indicates the information of the character string obtained by performing OCR processing on the character region specified by "rect". The information of this "rect" and "text" is included for all the character regions detected within the scan image to be analyzed. However, only a part of it is omitted in Fig. 15.

[0047] And, FIG. 16 shows the similar form determination result for the scan image of FIG. 13 obtained by accessing “matchingResultUrl”. In “matched”, “true” or “false” is entered as a value indicating whether a scan image similar to the scan image to be analyzed this time has been found among the past scan images by the above-described similar form determination process. If the value is “true”, it indicates that the form to be scanned this time is similar to the form scanned previously, and if it is “false”, it indicates that they are not similar. In “formId”, if a similar form is found, a value (ID value) that uniquely indicates the scan image of the past similar form is entered. On the other hand, if no similar form is found, a newly created ID value that uniquely indicates the scan image of the form scanned this time is entered. And this newly created ID value is used when learning the input information (file name configuration information) at the time of setting the file name for the scan image of a form for which no similar form existed in the past. In “matchingScore”, a value indicating the degree of match is entered if there is a similar form. The similarity in this embodiment is represented by a real number value from 0 to 1 indicating the degree of match between the arrangement information of the character regions in the past scan image and the arrangement information of the character regions in the current scan image, and the larger the value, the more similar the forms are. In “rectInfoArray”, information indicating the character regions in the scan image to be analyzed this time corresponding to the character regions used at the time of setting the file name for the scan image of the past similar form is entered.

[0048] In the case of the example in FIG. 16, previously, for the scanned image of FIG. 13 obtained by scanning a quotation, the file name was set using two character strings, "quotation" and "Shinagawa Co., Ltd.", and the learning of the input information was performed. After that, a quotation created in the same format was scanned to generate the scanned image shown in FIG. 17, and as a result of performing a similar form determination process on the scanned image, it was determined that it was similar to the scanned image of FIG. 13. And the example of FIG. 16 shows a state in which the information of the character area used when setting the file name for the scanned image of FIG. 13 is stored as the information of the character area to be automatically input. Hereinafter, the process until the result of the similar form determination process shown in FIG. 16 is obtained will be described.

[0049] Assume that the learning result based on the learning request in FIG. 11 described above is stored in the data management unit 434. In the similar form determination process, using the coordinate information of the character areas of "estimate" and "Shinagawa Co., Ltd." used at the time of setting the previous file name, a character area indicated by the coordinate information within the character areas included in the current scan image and a character area that partially overlaps it are identified. Then, the coordinate information of the character area identified as partially overlapping and the character string of the character area are stored in "text" of "rectInfoArray". A value that uniquely indicates the character area used for automatic input of the file name is entered in "key". In "region", the coordinate information of the character area uniquely indicated by "key" and the character string obtained by OCR processing for the character area are stored. And "rect" contains information for identifying the character area uniquely indicated by "key". In this case, "x" represents the X coordinate of the upper left corner of the character area, "y" represents the Y coordinate of the upper left corner of the character area, "width" represents the number of pixels in the X (width) direction of the character area, and "height" represents the number of pixels in the Y (height) direction of the character area. And the information of the character string obtained by performing OCR processing on the character area specified by "rect" is entered in "text". In "metadataArray", information indicating where the order of the character areas used when automatically inputting the file name and the delimiter character are entered is stored. When property information such as metadata other than the file name is set, the necessary information is added in "rectInfoArray" and "metadataArray". A value that uniquely indicates the setting value to be set in the scan image is stored in "key". A value indicating the type of the setting value of "key" is stored in "keyType". When used for the file name, "filename" is entered in "key" and "filename" is entered in "keyType". In "value", the information of the character area and the delimiter character used for the value of "key" is stored. In the example of FIG. 16, the character string of the character area having the "key" of "fileRegion0" in "rectInfoArray" is the first, the delimiter character is the second, and the character string of the character area having the "key" of "fileRegion1" is the third.Then, it will be automatically input into the file name input field 1001 in this order and displayed as the initial file name.

[0050] In S1209, the display control unit 421 displays the aforementioned file name setting screen 1000 on the operation unit 220 so that the user can set the file name for the scanned image. The details of this file name setting process will be described later.

[0051] In S1210, the cooperation service request unit 423 refers to the response to the request of the processing status in S1206 to determine whether the setting for automatic input of the file name is enabled, and determines the next process. Specifically, referring to the value of "autoInput" included in the response at the time of processing completion shown in FIG. 9(c) above, if it is "true" which means the setting for automatic input is enabled, it proceeds to S1211. On the other hand, if it is "false" which means the setting for not performing automatic input, it proceeds to S1214.

[0052] In S1211, the cooperation service request unit 423 determines whether there was a scanned image similar to the current scanned image in the past based on the image analysis result obtained in S1208. Specifically, referring to the value of "matched" included in the result of the similar form determination process shown in FIG. 16 above, if it is "true" indicating that there is a similar one among the past scanned images, it proceeds to S1212. On the other hand, if it is "false" indicating that there is no similar one, it proceeds to S1214.

[0053] In S1212, the cooperation service request section 423 determines whether the user has made a change to the initial file name automatically entered in the file name input field 1001 in the file name setting process of S1209. Changes in this case include, for example, making the character area used for the current file name different from the character area used in the initial file name, adding or deleting delimiter characters. This determination is made by comparing the value of "value" in "metadataArray" included in the result of the similar form determination process in FIG. 16 with the input information in the file name input field 1001 after setting the file name for the current scan image. If there is a difference as a result of the comparison, it is determined that a change has been made to the initial file name and the process proceeds to S1213. If they match, it is determined that no change has been made to the initial file name and the process proceeds to S1214. Regarding the leading character string of the file name automatically entered by the "value" of "formKeys", even if it is deleted by the user, the process may proceed to S1214 without considering that the input information has changed.

[0054] In S1213, the cooperation service request section 423 transmits a learning request for setting input information (refer to FIG. 11 above) representing the configuration of the file name set for the current scanned image to the MFP cooperation server 120. In the "formId" included in the learning request in this case, the value of "formId" included in the similarity determination processing result of the image analysis result obtained in S1208 is used. In the example of FIG. 16, since "aaaaaaaa-ffff-49ab-acf8-55558888eeee" is included as the value of "formId", this will be stored in the learning request. Also, the content of the input information used when setting the file name is stored in "learningContent". First, in "rectInfoArray", the coordinate information of the character area to which the character string used for the file name belongs is stored for the number of such character areas. Then, in "metadataArray", the information on the character area and the delimiter used for setting the file name is stored. Referring to Table 1 above, the operations performed by the user at this time will be described. In the preview area 1002 of the file name setting screen 1000, first touch the "estimate" with number 1. Next, touch the "Shinagawa Co., Ltd." with number 8, and finally press the send button 1007. The MFP cooperation server 120 that has received the learning request accumulates the input information based on such user operations and will use the accumulated input information in subsequent image analysis processing. After transmitting the learning request, when a response indicating that the learning request has been processed normally is received from the MFP cooperation server 120, it proceeds to S1214.

[0055] In S1214, the cooperation service request section 423 associates the scanned image data to be processed with the file name set in S1209 and transmits it to the MFP cooperation server 120. With the completion of this transmission, this process ends. In this embodiment, the transmission of the scanned image data and its file name is performed last, but it may be transmitted to the MFP cooperation server 120 in parallel at the time when the scanning process ends and the scanned image data is saved by the scanned image management section 412.

[0056] The above is the processing flow when using the storage service focusing on the operations in the MFP 110. In the above description, a scenario was assumed where the file name was set when a single scanned image obtained by scanning one estimate document was filed. However, the present embodiment is not limited to this, and it can be similarly applied to a scenario where the file name is set when filing a plurality of scanned images obtained by scanning a plurality of documents. In this case, buttons for turning pages may be provided within the file name setting screen 1000 so that the file name can also be set from the character areas of the scanned images on the second page, third page, etc.

[0057] (File name setting process) Subsequently, the file name setting process executed by the display control unit 421 in the aforementioned S1209 will be described in detail according to the flowcharts shown in FIGS. 18 and 19.

[0058] In S1801, a file name (initial file name) that is initially displayed in the file name input field 1001 of the aforementioned file name setting screen 1000 is generated. The initial file name is generated according to the following procedure. First, referring to the response at the completion of the aforementioned process, if there is any value in the "value" of "formKeys", that value is set at the beginning of the file name. For example, if the string "scan" is entered as the "value" of "formKeys", "scan" is set at the beginning of the initial file name. Next, a predetermined string is set according to the similar form determination result obtained by accessing and retrieving "matchingResultUrl". That is, when the value of "matched" is "true" and "matchingScore" is a numerical value equal to or greater than a certain value (for example, 0.6 or more), a string is set according to the file name configuration specified by the "value" of "metadataArray". Specifically, the character strings of the character regions specified by "rectInfoArray" are set in the order specified in the "value". In the example of the similar form determination result in FIG. 16, the value of "matched" is "true" and "matchingScore" is "0.74...", so it is a numerical value equal to or greater than a certain value. And in the "value" of "metadataArray", it is specified that the order is "fileRegion0" first, "separator" second, and "fileRegion1" last. Therefore, the value of "text" of "fileRegion0", which is "Estimate", the value of "separator" included in the process completion response, which is "_ (underscore)", and the value of "text" of "fileRegion1", which is "Shimomaru Co., Ltd.", are concatenated in order. Thus, "Estimate_Shimomaru Co., Ltd." is generated as the initial file name. If there is any value (for example, "scan_") in the "value" of "formKeys", "scan_Estimate_Shimomaru Co., Ltd.", with that character added at the beginning, becomes the initial file name.In the similar form determination result, when the value of "matched" is "false" and the value of "matchingScore" is less than a certain number, if the "value" of "formKeys" contains a string, that string will be used as the initial file name as it is. Also, if the "value" of "formKeys" does not contain a string, there will be no initial file name (blank). In this embodiment, two values, the value of "matched" and the value of "matchingScore", are used as the conditions for generating the initial file name based on the information specified in "rectInfoArray" according to the order of the "value" of "metadataArray". However, it is not limited to this, and only one of the results may be used as the condition. For example, the initial file name may be generated only on the condition that the value of "matched" is "true", or the file name may be generated only on the condition that the value indicating the degree of similarity is equal to or greater than the threshold value.

[0059] In S1802, when the scan image is initially displayed in the preview area 1002 of the file name setting screen 1000, the degree of reduction for display (hereinafter referred to as the "initial magnification ratio") is determined. To determine this initial magnification ratio, the character recognition result obtained by accessing "ocrResultUrl" is used. First, the leftmost character area and the rightmost character area among all the character areas detected in the scan image are obtained respectively. In this case, the leftmost character area is the character area with the minimum x - coordinate value among all the character areas. Also, the rightmost character area is the character area for which the sum of the x - coordinate value and the character area width (width) value is the maximum among all the character areas. The magnification ratio such that the left side of the leftmost character area thus determined coincides with the left side of the preview area 1002 and the right side of the rightmost character area coincides with the right side of the preview area 1002 is determined as the initial magnification ratio. Note that if the edge of the determined character area is made to exactly coincide with the edge of the preview area, it may be difficult for the user to select (press) when the character area is small. Therefore, the initial magnification ratio may be determined with a certain amount of margin. For example, the initial magnification ratio is determined so that a margin of 10 pixels is ensured from the left end of the character area that will be displayed most leftward and the right end of the character area that will be displayed most rightward within the preview area 1002. Also, when UI components such as various operation buttons are superimposed and displayed on the preview area 1002, the initial magnification ratio may be determined so that the character area does not overlap with the UI component. Specifically, after subtracting the width of the UI component from the width of the preview area 1002, the initial magnification ratio may be determined so that the leftmost character area and the rightmost character area fit within the width of the remaining area. In the example of the file name setting screen 1000 in FIG. 10 mentioned above, since there are three operation buttons (enlargement button 1004, reduction button 1005, return button 1006) in the preview area 1002, the initial magnification ratio is determined so that the character area does not overlap with these buttons.In addition, header information related to printing processing may be printed near the upper, lower, left, and right ends of the document to be scanned. The possibility that the character string in the character area corresponding to such header information, etc. is adopted as part of the file name is low. Therefore, character areas existing within a certain amount (for example, 50 pixels) from the upper, lower, left, and right ends of the scanned image may be excluded when obtaining the leftmost and rightmost character areas described above.

[0060] In S1803, it is determined which part of the scanned image after scaling by the initial magnification factor determined in S1802 is to be initially displayed within the preview area 1002 of the file name setting screen 1000. In the scanned image scaled by the magnification factor determined in S1802 above, the width within which the leftmost character area and the rightmost character area fit is substantially the same as the width of the preview area 1002. Therefore, first, for the horizontal direction, the part to be initially displayed (hereinafter referred to as the "initial display part") is determined so that these left and right character areas just fit within the preview area 1002. For example, the left side of the leftmost character area coincides with the left side of the preview area 1002. Then, the initial display part in the vertical direction is determined so that the uppermost character area is positioned near the upper end of the preview area 1002. At this time, if margins were considered when determining the initial magnification factor, the initial display part may be determined in consideration of the margin in the same way. For example, character areas existing within a certain amount (e.g., 50 pixels) from the upper end of the scanned image may be excluded from the display target, and it may be determined that the uppermost character area among the other character areas is displayed at the upper end of the preview area 1002. FIG. 20 is a diagram showing the initial display part determined for the scanned image of FIG. 13. The line frame in FIG. 20 indicates the range of the scanned image initially displayed in the preview area 1002. Note that as a result of determining the initial display part as described above, there may be a case where the character area of the character string used for the initial file name when similar forms exist and the initial file name is automatically entered in the file name input field 1001 is not displayed in the preview area 1002. In this case, for example, the initial display part may be determined so that the character area of the character string used for the initial file name is displayed in the center of the preview area 1002. At this time, the character string constituting the initial file name automatically entered in the file name input field 1001 and the character area of the character string used for the initial file name may each be highlighted (e.g., by changing the color). FIG. 21 is an example of the scan setting screen 1000 when the character string "Kawasaki Corporation" is automatically entered in the initial file name in the scanned image of FIG. 13.Here, so that the character area corresponding to the character string "Kawasaki Corporation" is located at the center of the preview area 1002, further, the color of the character area and the character string "Kawasaki Corporation" entered in the file name input field 1001 is changed and displayed. At this time, when the character size of the character area to be highlighted is small, the initial magnification ratio may be adjusted so as to obtain an appropriate character size. The initial magnification ratio after this adjustment can be obtained based on, for example, the size of the touch panel of the operation unit 220 and the range displayed in the preview area 1002 in the initial state. Specifically, based on the height of the character area of the character string used for the initial file name, the character size in points is obtained, and the initial magnification ratio of the preview is adjusted so that the obtained points are equal to or greater than the threshold value (predetermined size or more).

[0061] In S1804, using the processing results from S1801 to S1803, the aforementioned file name setting screen 1000 is displayed on the operation unit 220. The user sets the file name for the scanned image to be stored in the storage server 130 using the file name setting screen 1000 illustrated in FIG. 10. This will be described in detail below.

[0062] 1) File name input field When the user touches the file name input field 1001, the display of the touch panel of the operation unit 220 switches to the soft keyboard, and the user can input arbitrary characters or edit the automatically input character string. For example, when touching the character string portion of the file name input field 1001 in the state where "Kawasaki Corporation" is automatically input as in the example of FIG. 21 described above, the soft keyboard 2200 shown in FIG. 22 is displayed. At this time, as shown in FIG. 22, the character string touched in the input area 2201 is displayed in the input state. Thereby, the user can operate the soft keyboard 2200 to correct the touched character string "Kawasaki Corporation". Further, when the file name input field 1001 in the state where no character string is input as shown in FIG. 10 is touched, the soft keyboard 2200 in the state where the input area 2201 is not input is displayed, and the user can operate this to input an arbitrary character string into the file name input field 1001.

[0063] 2) Preview area As described above, a specific part of the magnified scan image is displayed in the preview area 1002. When any character area within the preview area 1002 is touched and selected, the character string of the selected character area is added to the file name input field 1001. At this time, in order to indicate that it has been selected, the character area related to the selection may be highlighted by underlining, surrounding it with a frame, coloring it, etc. Also, the display part may be changed so that the selected character area is centered or the display magnification may be changed. Further, when a plurality of character areas are selected, the display mode may be made different for each character area, such as coloring each character area with a different color. Then, when the user touches the selected character area again, the selection state is cancelled (that is, the lines or colors attached to the character area are deleted and returned to the original state), and the character string corresponding to the character area in the file name input field 1002 is also deleted. Note that in this embodiment, when the character area is not selected, frames indicating each character area are not displayed in the preview area 1002. However, frames indicating each character area may be displayed in advance so that the user can recognize the touchable part, and the color may be changed when it is selected. Also, the user may be able to switch whether to perform a display that makes each character area distinguishable by a button or the like. And the user can move the displayed image part by performing a swipe operation in the preview area 1002.

[0064] 3) Various operation buttons The button 1003 with an "×" mark is a delete button for deleting the last character string among the character strings being input in the file name input field 101. The button 1004 with a "+" mark is a magnification button for magnifying and displaying the image being displayed in the preview area 1002, and is used when it is desired to display each character area larger. The button 1005 with a "-" mark is a reduction button for reducing and displaying the image being displayed in the preview area 1002, and is used when it is desired to display a wider range. When performing magnification or reduction, the display position is adjusted so that the center coordinates of the preview area 1002 are the same as before the magnification or reduction. The button 1006 with a "U-turn arrow" mark is a return button for returning the preview area 1002 to the initial display state when moving the display position by a swipe operation or changing the display magnification by pressing the magnification button 1004 or the reduction button 1005. The "Send" button 1007 is a button for filing the scanned image with the file name set on the file name setting screen 1000 and saving it to the storage server 130. In response to the pressing of this "Send" button 1007, the scanned image data is transmitted to the MFP cooperation server 120 together with the information of the set file name.

[0065] In S1805, operation guidance is displayed within the file name setting screen 1000 in the initial display state. Through this operation guidance, even a user who is operating for the first time can easily understand that the file name can be set by touching and selecting the character area within the preview area 1002. FIG. 23 shows a state where a message 2301 as operation guidance is superimposed and displayed on the upper part of the preview area 1002 in the file name setting screen 1000 shown in FIG. 10. Such operation guidance is unnecessary for users who already know the operation method. Therefore, when any user operation such as a touch operation or a swipe operation is detected, the operation guidance is hidden. Also, at the upper part of a general document, there are often characters and numbers that have important meanings for the document, such as a document title, company name, form number, and date. Therefore, in order to prevent the character area of these strings with important meanings from overlapping with the operation guidance, the image part displayed in the preview area 1002 may be shifted downward according to the attributes of the string. Also, when the operation guidance overlaps with any character area or enters within a certain distance from the character area, the display position may be determined so as not to overlap or get too close. In this way, by flexibly determining the display position of the operation guidance without fixing it, it becomes easier for the user to select the character area.

[0066] S1806 is a monitoring process for the presence or absence of user operations. It checks for the presence or absence of user operations at a predetermined interval, and if any user operation is detected, it proceeds to S1807. Then, in S1807, as described above, the operation guidance is hidden.

[0067] S1808 to S1815 are determination processes that determine the next process according to the content of the detected user operation. Note that when the process corresponding to the detected user operation is completed, it returns to S1806 and waits for the next user operation to be detected.

[0068] In S1808, it is determined whether the detected user operation is a touch operation on any character area within the preview area 1002. Specifically, first, the coordinates of the touched position are converted in association with the coordinates of the scanned image. Then, if the converted coordinates are included within any of the regions specified by the "regions" in the above-described character recognition result (refer to FIG. 15), it is determined that it is a touch on the character area. As a result of the determination, if it is a touch on the character area, the process proceeds to S1816, and if it is not a touch on the character area, the process proceeds to S1809. In S1816 when it is a touch on the character area, it is determined whether the touched character area has already been selected. If the touched area is an unselected character area, the process proceeds to S1817, and if the touched area is a selected character area, the process proceeds to S1818.

[0069] In S1817, the character string of the touched character area is added to the file name input field 1001, and highlighting processing indicating that it has been selected is performed on the character area. At this time, if no other character string is input in the file name input field 1001, the character string is added as it is. Then, the fact that the character area has been used for file name setting is stored as input information at the time of file name setting. On the other hand, if a character string of another character area has already been input in the file name input field 1001, after a delimiter character is added, the character string of the touched character area is added. In this case, in addition to the fact that the character area has been used for file name setting, the fact that a delimiter character has been inserted between the character strings, such as "the first character area", "delimiter character", "the second character area", is also stored. FIG. 24 is a diagram showing the file setting screen 1000 when the character area recognized as "estimate" is selected first, and subsequently the character area recognized as "Shimomaruko Co., Ltd." is selected. In the file name input field 1001, two character strings corresponding to the order of the touched character areas are input, and an underscore as a separator is input between the two character strings.

[0070] In S1818, the character string in the touched character area is deleted from the file name input field 1001, the highlighting indicating that the character area has been selected is canceled, and it returns to the normal display state indicating that it is not selected. Also, if there is a delimiter character attached to the character string, the delimiter character is also deleted together. For example, assume that the file name configuration used for generating the initial file name is ["the first character area", "delimiter character", "the second character area"]. In this case, if the user touches the character area that is "the first character area", the delimiter character is also deleted together with the character string of the first character area, and only the character string of "the second character area" remains in the file name input field 1001.

[0071] In S1809 when the detected user operation is not a touch on a character area, it is determined whether the detected user operation is a swipe operation within the preview area 1002. If a swipe operation is detected, it proceeds to S1819; otherwise, it proceeds to S1810.

[0072] In S1819, a process is executed to move the image portion displayed in the preview area 1002 in the swiped direction. Specifically, the image portion displayed in the preview area 1002 is updated according to the amount of movement of the user's finger on the touch panel. This process may be limited to when a touch operation in an area other than the character area within the preview area 1002 is detected. Also, even if it is a touch operation on a character area, it may be treated as a swipe operation according to the amount of movement of the user's finger afterwards instead of being treated as a touch on the character area.

[0073] In S1810, it is determined whether the detected user operation is a press of the zoom-in button 1004 or the zoom-out button 1005. If it is a press of the zoom-in button 1004 or the zoom-out button 1005, it proceeds to S1820; otherwise, it proceeds to S1811.

[0074] In S1820, a process of enlarging or reducing the image portion being displayed in the preview area 1002 is executed. Specifically, when the enlargement button 1004 is pressed, the magnification of the scanned image is increased by a certain amount, and when the reduction button 1005 is pressed, the magnification of the scanned image is decreased by a certain amount. The certain amount here is arbitrary, but for example, it is about 10% based on the current magnification. At this time, if there is an initial magnification or a magnification at which the width / height of the scanned image matches the width / height of the preview area 1002 before performing a certain amount of enlargement or reduction, it may be stopped at that magnification. Note that it is controlled so that it cannot be reduced below the magnification (minimum magnification) at which the entire scanned image fits within the preview area 1002, and cannot be enlarged beyond the magnification (maximum magnification) at which the smallest character area becomes a certain size.

[0075] In S1811, it is determined whether the detected user operation is a press of the return button 1006. If it is a press of the return button 1006, the process proceeds to S1821, and otherwise, it proceeds to S1812.

[0076] In S1821, a process of returning the preview area 1002 to the initial display state is executed. Specifically, the magnification of the scanned image to be displayed in the preview area 1002 is returned to the initial magnification determined in S1802, and further, the display portion in the preview area 1002 is returned to the initial display portion determined in S1803. At this time, if the initial magnification or the initial display portion has been adjusted, it is desirable to return to the adjusted initial magnification and initial display portion.

[0077] In S1812, it is determined whether the detected user operation is a touch operation on the character string input in the file name input field 1001. At this time, if a touch operation on the blank area after the character string input in the file name input field 1001 is detected, it may be treated as a touch operation on the last character string. If it is a touch operation on the character string input in the file name input field 1001, the process proceeds to S1822, and otherwise, it proceeds to S1813.

[0078] In S1822, an editing soft keyboard is displayed on the operation unit 220 in a state where the character string subjected to a touch operation is input to the input area 2201. Now, assume that the file name setting screen 1000 is in the state shown in FIG. 24. In this state, if a touch operation on the character string "Estimate" in the file name input field 1001 is detected, the soft keyboard 2200 in a state where the character string "Estimate" is input to the input area 2201 of the soft keyboard is displayed. Further, if a touch operation on the character string "Shimamaruko Co., Ltd." or a touch operation on the blank area behind it is detected, the soft keyboard 2200 in a state where the character at the end of the character string, i.e., "_Shimamaruko Co., Ltd.", is input to the input area 2201 is displayed. Note that since the operation of the soft keyboard 2200 is the same as that of a general keyboard, its description is omitted. The subsequent S1823 is a monitoring process for the presence or absence of user operations. It checks at a predetermined interval whether the OK button 2202 on the soft keyboard 2200 has been pressed, and if the pressing of the OK button 2202 is detected, it proceeds to S1824. Note that if a key other than the OK button is pressed, the character string displayed in the input area 2201 is edited according to the pressed key. For example, the user can use the soft keyboard 2200 to correct a character string that has been misrecognized in the character recognition process.

[0079] In S1824, it is determined whether a change to delete the delimiter character has been made. In the present embodiment, this determination is made only when the character string touched in S1812 is displayed together with the delimiter character in S1822, and for a character string that was not displayed together with the delimiter character, it is treated as if the delimiter character has not been deleted. That is, when the delimiter character is added to the beginning of the character string and displayed in S1822, when the OK button 2202 is pressed, it is determined whether an edit of the character string to delete the delimiter character has been made based on whether the delimiter character exists at the beginning of the character string. For example, when a touch operation is performed on the character string "Shimamaruko Co., Ltd." in S1812, as described above, the input area 2201 of the soft keyboard 2200 is displayed in the state of "_Shimamaruko Co., Ltd." with the delimiter character attached to the beginning. Therefore, by checking whether the delimiter character remains at the beginning, it is determined whether the delimiter character has been deleted. Also, when a touch operation is performed on the character string "Estimate" in S1812, since "Estimate" is displayed in the input area 2201 as described above, in this case, it is treated as if no change to delete the delimiter character has been made. In the above example, it has been described that when the delimiter character exists before the character string touched in S1812, in S1822, the character string with the delimiter character attached to the beginning is displayed in the input area 2201, but it is not limited to this. For example, when the delimiter character exists after the character string touched in S1812, in S1822, when it is configured to display the delimiter character attached after the touched character string in the input area 2201, the same determination can be made in S1824. That is, an example in FIG. 24 will be described where, in response to a touch operation on the character string "Estimate", the input area 2201 of the soft keyboard 2200 is configured to display with the delimiter character attached to the end such as "Estimate_". In this case, in this step S1824, it may be determined whether the delimiter character has been deleted by checking whether the delimiter character remains at the end.

[0080] If it is determined in S1824 that the delimiter character has been deleted, proceed to S1825. Then, in S1825, it is remembered that the delimiter character has been deleted from the file name. For example, assume that the file name configuration used for generating the initial file name is ["First character area", "Delimiter character", "Second character area"]. At this time, if the character string corresponding to the "Second character area" in the file name input field 1001 is touched and the delimiter character is deleted using the soft keyboard 2200, the file name configuration after the change will be ["First character area", "Second character area"]. And the information of this file name configuration after the change will be retained. In the subsequent S1826, it is determined whether a change has been made to add a delimiter character to the file name. If it is determined that the delimiter character has been added, proceed to S1827. Then, in S1827, it is remembered that the delimiter character has been added to the file name. For example, assume that the file name configuration used for generating the initial file name is ["First character area", "Second character area"]. At this time, if the character string corresponding to the "Second character area" in the file name input field 1001 is touched and the delimiter character is added at the beginning using the soft keyboard 2200, the file name configuration after the change will be ["First character area", "Delimiter character", "Second character area"]. And the information of this file name configuration after the change will be retained.

[0081] Then, in S1828, after the display screen of the operation unit 220 is returned from the soft keyboard 2200 to the file name setting screen 1000, the file name is corrected to the modified content. At this time, the display portion of the scanned image may be changed so that the character area corresponding to the modified character string comes to the center of the preview area 1002. Even if a part of the character string used in the file name is corrected, if the file name configuration has not changed, the information on the file name configuration is not newly retained. However, when a correction is made to delete all the characters included in a certain character string, the file name configuration also changes. Therefore, also in this case (although omitted in the flowchart), the configuration information of the changed file name is retained. For example, assume that the file name configuration used for generating the initial file name is ["first character area", "delimiter character", "second character area"]. At this time, assume that the character string corresponding to the "second character area" in the file name input field 1001 is touched and all the characters are deleted using the soft keyboard 2200. In this case, the unnecessary delimiter character is also deleted and the changed file name configuration changes to ["first character area"], so the information on the changed file name configuration is retained.

[0082] In S1813, it is determined whether the detected user operation is a touch operation on the empty file name input field 1001 in a state where no character string is input. If it is a touch operation on the empty file name input field 1001, the process proceeds to S1829; otherwise, the process proceeds to S1814.

[0083] In S1829, a soft keyboard for adding a character string in a state where nothing is input in the input area 2201 is displayed on the operation unit 220. The subsequent S1830 is a monitoring process for the presence or absence of a user operation. The presence or absence of pressing the OK button 2202 on the soft keyboard 2200 is checked at a predetermined interval, and if the pressing of the OK button 2202 is detected, the process proceeds to S1831. Then, in S1831, after the display screen of the operation unit 220 is returned from the soft keyboard 2200 to the file name setting screen 1000, the input character string is added to the file name.

[0084] In S1814, it is determined whether the detected user operation is the pressing of the delete button 1003. If it is the pressing of the delete button 1003, the process proceeds to S1832; otherwise, it proceeds to S1815. In S1832, the last character string among the character strings input in the file name input field 1001 is deleted. For example, if the file name configuration used for generating the initial file name is ["the first character area", "delimiter character", "the second character area"], the character string corresponding to "the second character area" is deleted by pressing the delete button 1003. As a result, the file name configuration after deletion becomes ["the first character area 1"], and the file name becomes only the character string corresponding to "the first character area". At this time, the character area corresponding to the deleted character string is returned to a non-selected state in the preview area 1002. And the information on the modified file name configuration is retained.

[0085] In S1815, it is determined whether the detected user operation is the pressing of the send button 1007. If it is the pressing of the send button 1007, this process ends and the process proceeds to S1210 in the flowchart of FIG. 12.

[0086] The above is the content of the file name setting process executed in S1209.

[0087] <Modification Example 1> In the above S1802 (Determination of Initial Magnification Ratio), if, according to the initial magnification ratio determined based on the leftmost character area and the rightmost character area, the average height of each character area displayed in the preview is less than a predetermined threshold value, the initial magnification ratio may be changed so that it is equal to or greater than the predetermined threshold value. In this case, the "average height" may be the average for all character areas or the average for some character areas. Examples of the average for some character areas include the average height of the character areas that will be displayed in the preview area 1002 at the determined initial magnification ratio, or the average height of the character areas in a specific part (for example, the upper half) of the scanned image. A specific example will be described. Now, assume that the distance (width) between the left end of the leftmost character area and the right end of the rightmost character area is 2000 px, the width of the preview area 1002 is 800 px, and the predetermined threshold value is 15 px. Here, when the average height of all character areas is 60 px, the initial magnification ratio calculated is 0.4 (= 800 ÷ 2000), and since the average height of all character areas is 24 px (= 60 × 0.4) and is not less than the predetermined threshold value, 0.4 remains the initial magnification ratio. On the other hand, when the average height of all character areas is 30 px, the average height of all character areas becomes 12 px (= 30 × 0.4). In this case, since it is less than the predetermined threshold value, the initial magnification ratio is changed to 0.5 (= 0.4 × 15 ÷ 12) so that the average height of all character areas is equal to or greater than the predetermined threshold value (here, the same value as the threshold value). Fig. 25 shows an example of the initial display of the scan setting screen 1000 when the average height of the character areas fails to meet the threshold value and the initial magnification ratio is adjusted. In this example, after adjusting the value of the initial magnification ratio to be larger, in S1803, the display part is determined in alignment with the left end of the leftmost character area and the upper end of the uppermost character area. As a result, the area corresponding to the right side of the form is not included in the display part, but the visibility of each character area existing within the preview area 1002 is ensured.

[0088] <Modification Example 2> In the above-mentioned S1803 (determination of the initial display part), when there are multiple character areas used for the initial file name, the character area at the top among the used character areas may be displayed in the preview area 1002. Also, the semantic type of the character string in each character area may be determined, and the display part may be adjusted so that the character area with a high possibility of being used for the file name is displayed within the preview area 1002. Further, the display part may be determined so that the number of character areas specified in advance are displayed in the preview area 1002. Additionally, the display part may be determined so that the midpoint between the character area at the top and the character area at the bottom among the character areas of the character string used for the initial file name is at the center of the preview area 1002.

[0089] <Modification Example 3> In the above-mentioned S1805 (display of operation guidance), when the initial file name according to the file name configuration of similar forms is automatically input in the file name input field 1001, it may be displayed so as not to overlap with the character areas of the character string used for the initial file name. At this time, there may be a case where there are multiple character strings used for the initial file name and it is difficult to display them so that they do not overlap with those character areas. In such a case, the character area of the character string at the beginning of the initial file name may be prioritized and displayed overlapping a part or all of the character area of the character string at the end. Also, when setting document properties other than the initial file name, such as metadata, etc., the character areas that avoid overlapping of the operation guidance may be determined according to the type of the document property, and the display position of the operation guidance may be determined. For example, when setting a value for a document property of date type, the operation guidance is displayed so as not to overlap with the character area of the date in the target document. At this time, whether the character string corresponding to each character area is a date is determined by whether the character string extracted from the scanned image fits a specific character array. For example, in the format of yy / mm / dd, when numbers are entered for each of "yy", "mm", and "dd", it can be determined that it is a date.

[0090] <Modification Example 4> In the above S1822 (display of the editing soft keyboard), instead of displaying the touched character string in the input area 2201 in the input state, the entire file name may be displayed in the input state. In this case, in order to prevent the user from misidentifying the editing target, highlighting display (changing the color, adding a frame, etc.) for distinguishing from other character strings may be performed, or the cursor may be displayed at the end thereof, so as to display in a manner that the range of the editing target can be understood. Also, instead of displaying the editing soft keyboard in response to a touch operation on the character string input in the file name input field 1001, the editing soft keyboard may be displayed after waiting for an additional user operation (for example, pressing a dedicated button not shown).

[0091] <Modification Example 5> Also, in the description of this embodiment, when a touch operation by the user on the character area is detected, processing (S1817, S1818) for adding or deleting the character string of the touched character area to the file name is executed. However, the response of the system may be slow, and it may take time from detecting the touch operation until the addition or deletion of the target character string is reflected in the file name. In this case, the user may misunderstand that the system is not reacting to his / her touch operation and may repeat the touch operation on the same character area many times. In such a case, the addition and deletion of the character string in the same character area may be repeatedly performed, and the user may not be able to stop in the desired state. For example, when the character string to be deleted is touched multiple times, the character string to be deleted may be re-entered. Therefore, in order to prevent the touch operation on the same character area from being repeatedly performed during the addition and deletion processing, new addition and deletion touch operations on the same character area may not be accepted until the addition and deletion processing is completed. At this time, a message indicating that the processing is in progress may also be displayed. Thereby, it is possible to prevent the addition and deletion of character strings unintended by the user from being executed during the input operation when setting the file name.

[0092] (Other Embodiments) The present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, an ASIC) that implements one or more functions.

Claims

Claim 1 detection means for detecting a character region from a scanned image obtained by scanning a document; display control means for displaying a UI screen for setting properties for the scanned image; comprising; the UI screen has a preview area for displaying the scanned image; the display control means; determines a predetermined magnification based on the character region detected by the detection means; stores the determined predetermined magnification; performs an initial display in the preview area using the scanned image magnified or reduced based on the predetermined magnification; after the preview area has changed from the initial display state, in response to detection of an operation of a predetermined button present on the UI screen, returns the preview area to the initial display state based on the stored magnification; An image processing apparatus characterized by the above.

Images