Information processing apparatus, control method thereof, and storage medium

Abstract

The present invention relates to an information processing apparatus, a control method thereof, and a storage medium. A user can easily perform partial editing of an undetermined file name or the like being displayed on a UI screen of an image processing apparatus. An editing UI screen using a character string selected by the user as a target is displayed.

Description

[0001] (This application is a divisional application of the application filed on April 17, 2020, with application number 2020103056733, entitled "Image Processing Apparatus and Control Method Thereof and Storage Medium Thereof".) Technical Field

[0002] The technology of this invention relates to character input technology. Background Technology

[0003] Traditionally, a widely used document management method involves transforming a scanned image obtained by scanning a document into a file of a predetermined format and sending the file to a storage server on a network for storage. Then, as a simple method for setting filenames for scanned images, there exists a method that allows users to select a character area (text block) on the scanned image displayed in a preview. Japanese Patent Application Laid-Open No. 2018-124656 discloses a technique that, when a scanned image obtained by scanning a document matches predefined image features, generates and displays a filename using a string described at a specific location within the document, and allows the user to edit the filename.

[0004] In the aforementioned Japanese Patent Application Publication No. 2018-124656, multiple strings obtained from a specific location in a document are concatenated and presented to the user as a series of filenames (displayed on a UI screen). Therefore, a user wishing to edit individual strings that are part of the filenames needs to specify the position and range of the string to be edited within the entire presented filename. In other words, partial editing of the presented filenames requires a significant amount of time and effort.

[0005] To address the aforementioned problems, the technology of this invention was developed. In other words, the aim is to enable users to easily partially edit undefined filenames and other elements displayed on the UI screen. Summary of the Invention

[0006] An image processing apparatus according to the present invention includes: detecting a character region from a scanned image obtained by scanning a document; controlling the display of a UI screen for setting attributes of the scanned image, wherein the displayed UI screen includes an input field for displaying a string corresponding to a selected character region as an attribute to be set, wherein, upon detecting an operation on the input field which is in a state displaying at least one string, a soft keyboard is displayed for modifying a string specified from the at least one displayed string based on the operation; and after modifying the specified string using the soft keyboard, setting attributes of the scanned image by using the string displayed in the input field.

[0007] An image processing apparatus includes: at least one memory for storing a program; and at least one processor for executing the program to: detect character regions from a scanned image obtained by scanning a document; control the display of a UI screen for setting attributes of the scanned image, wherein the displayed UI screen includes an input field for displaying a string corresponding to a selected character region as an attribute to be set, wherein, upon detection of an operation on the input field which is in a state displaying at least one string, a soft keyboard is displayed for modifying a string specified from the at least one displayed string based on the operation; and after modifying the specified string using the soft keyboard, the attributes of the scanned image are set by using the string displayed in the input field.

[0008] A control method for an image processing device, the control method comprising: detecting a character region from a scanned image obtained by scanning a document; controlling the display of a UI screen for setting attributes of the scanned image, wherein the displayed UI screen includes an input field for displaying a string corresponding to a selected character region as an attribute to be set; and wherein, upon detecting an operation on the input field which is in a state of displaying at least one string, a soft keyboard is displayed for modifying a string specified from the at least one displayed string based on the operation; and after modifying the specified string using the soft keyboard, setting attributes of the scanned image by using the string displayed in the input field.

[0009] A non-transitory computer-readable storage medium storing a program for causing a computer to: detect character regions from a scanned image obtained by scanning a document; control the display of a UI screen for setting attributes of the scanned image, wherein the displayed UI screen includes an input field for displaying a string corresponding to a selected character region as an attribute to be set, and wherein, upon detection of an operation on the input field which is in a state of displaying at least one string, a soft keyboard is displayed for modifying a string specified from the at least one displayed string based on the operation; and after modifying the specified string using the soft keyboard, the attributes of the scanned image are set by using the string displayed in the input field.

[0010] Further features of the invention will become apparent from the following description of typical embodiments with reference to the accompanying drawings. Attached Figure Description

[0011] Figure 1 This is a diagram showing the overall structure of the image processing system;

[0012] Figure 2 This is a block diagram illustrating the hardware structure of an MFP;

[0013] Figure 3 This is a block diagram showing the hardware structure of the MFP collaboration server and storage server;

[0014] Figure 4 This is a block diagram illustrating the software structure of an image processing system;

[0015] Figure 5 It is a sequence diagram showing the processing flow of the entire image processing system;

[0016] Figure 6 This is an example image showing the main menu screen;

[0017] Figure 7 This is an example image showing the login screen;

[0018] Figure 8 This is an example image showing the scan settings screen;

[0019] Figure 9A This is a diagram showing an example of a request ID, and Figure 9B and Figure 9C Each is a diagram illustrating an example of a response to a query related to a processing situation;

[0020] Figure 10 This is an example image showing the filename settings screen;

[0021] Figure 11 This is a diagram illustrating an example of a learning request;

[0022] Figure 12 This is a flowchart illustrating the processing flow in MFP;

[0023] Figure 13 This is a diagram showing an example of a scanned image;

[0024] Figure 14 This is an example image showing the filename condition settings screen;

[0025] Figure 15 This is a diagram showing an example of character recognition results;

[0026] Figure 16 This is a diagram illustrating an example of the judgment results for similar business forms;

[0027] Figure 17 This is a diagram showing an example of a scanned image;

[0028] Figure 18A and 18BThis is a flowchart illustrating the filename setting process in MFP;

[0029] Figure 19A and 19B This is a flowchart illustrating the filename setting process in MFP;

[0030] Figure 20 This is a diagram showing the initial display portion of the preview area;

[0031] Figure 21 This is an example image showing the filename settings screen;

[0032] Figure 22 This is a diagram showing an example of a soft keyboard;

[0033] Figure 23 This is an example of a filename settings screen that displays operation guidance;

[0034] Figure 24 This is an example diagram showing the filename setting screen when the character area is touched; and

[0035] Figure 25 This is a diagram showing the initial display portion of the preview area when the initial magnification is adjusted. Detailed Implementation

[0036] The present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments. The structures shown in the following embodiments are merely exemplary, and the invention is not limited to the structures illustrated schematically.

[0037] [First Embodiment]

[0038] <System Structure>

[0039] Figure 1 This is a diagram illustrating the overall structure of the image processing system according to this embodiment. The image processing system includes an MFP (Multifunction Peripheral Device) 110 and server devices 120 and 130 that provide cloud services over the Internet. The MFP 110 is connected to the server devices 120 and 130 to enable communication via the Internet.

[0040] The MFP 110 is an example of an information processing device having a scanning function. The MFP 110 is a multi-functional peripheral device having multiple functions such as a printing function and a BOX (box) storage function in addition to the scanning function. Each of the server devices 120 and 130 is an example of an information processing device that provides a cloud service. The server device 120 of the present embodiment provides a cloud service to perform image analysis on a scanned image received from the MFP 110, and to transmit a request from the MFP 110 to the server device 130 that provides other services, and so on. Hereinafter, the cloud service provided by the server device 120 is referred to as "MFP collaboration service". The server device 130 provides a cloud service (hereinafter referred to as "storage service") to save a file transmitted via the Internet, and to provide a saved file in response to a request from a web browser of a mobile terminal (not shown) or the like, and so on. In the present embodiment, the server device 120 that provides the MFP collaboration service is referred to as "MFP collaboration server", and the server device 130 that provides the storage service is referred to as "storage server".

[0041] Figure 1 The structure of the illustrated image processing system 100 is an example, and the structure is not limited thereto. For example, the MFP 110 may also have the function of the MFP collaboration server 120. In addition, the MFP collaboration server 120 may be connected to the MFP 110 via a LAN (local area network) instead of the Internet. In addition, the storage server 130 may be replaced with a mail server that performs a mail distribution service, and the system may be applied to a case where a scanned image of a document is transmitted by attaching the scanned image to a mail.

[0042] <Hardware Structure of MFP>

[0043] Figure 2This is a block diagram illustrating the hardware structure of the MFP 110. The MFP 110 includes a control unit 210, an operation unit 220, a printer unit 221, a scanner unit 222, and a modem 223. The control unit 210 includes various units 211-219 and controls the operation of the entire MFP 110. The CPU 211 reads various control programs (programs corresponding to various functions shown in the software structure diagram described later) stored in the ROM 212 and executes these programs. The RAM 213 serves as a temporary storage area for the CPU 211, such as main memory and working area. In this embodiment, one CPU 211 performs the various processes shown in the flowchart described later using a single memory (RAM 213 or HDD 214), but this embodiment is not limited to this. For example, the various processes can also be performed by having multiple CPUs and multiple RAMs or HDDs cooperate with each other. The HDD 214 is a large-capacity storage unit configured to store image data and various programs. Operation unit I / F 215 is the interface connecting operation unit 220 and control unit 210. Operation unit 220 is equipped with a touch panel and keyboard, and receives user operations / inputs / instructions. Touch operations on the touch panel include operations using a human finger and operations using a stylus. Printer I / F 216 is the interface connecting printer unit 221 and control unit 210. Image data for printing is transmitted from control unit 210 to printer unit 221 via printer I / F 216 and printed on a printing medium such as paper. Scanner I / F 217 is the interface connecting scanner unit 222 and control unit 210. Scanner unit 222 optically reads a document placed on an unshown document tray or ADF (Automatic Document Feeder) to generate scanned image data and inputs this data to control unit 210 via scanner I / F 217. The printer unit 221 can print (copy and output) scanned image data generated by the scanner unit 222, store the data in the HDD 214, and send the data as a file to external devices such as the MFP collaboration server 120, etc. The modem I / F 218 is the interface connecting the modem 223 and the control unit 210. The modem 223 communicates image data with a fax machine (not shown) on the PSTN via fax. The network I / F 219 is the interface connecting the control unit 210 (MFP 110) to the LAN. The MFP 110 sends scanned image data to the MFP collaboration server 120 and receives various data from the MFP collaboration server 120 using the network I / F 219, etc. The hardware structure of the MFP 110 described above is an example, and the hardware structure may include other structures as needed, or may not include any part of this structure.

[0044] <Hardware Structure of Server Device>

[0045] Figure 3 is a block diagram showing the hardware structure of the MFP Collaboration Server 120 / Storage Server 130. The MFP Collaboration Server 120 and the Storage Server 130 have a common hardware structure including a CPU 311, a ROM 312, a RAM 313, a HDD 314, and a Network I / F 315. The CPU 311 controls the entire operation by reading the control program stored in the ROM 312 and performing various processes. The RAM 313 is used as a temporary storage area such as a main memory and a work area for the CPU 311. The HDD 314 is a large-capacity storage unit configured to store image data and various programs. The Network I / F 315 is an interface that connects the control unit 310 to the Internet. The MFP Collaboration Server 120 and the Storage Server 130 receive requests for various processes from other devices (such as the MFP 110) via the Network I / F 315 and return processing results corresponding to these requests.

[0046] <Software Structure of Image Processing System>

[0047] Figure 4 is a block diagram showing the software structure of the image processing system 100 according to the present embodiment. Hereinafter, the software structure corresponding to the respective roles of the MFP 110, the MFP Collaboration Server 120, and the Storage Server 130 constituting the image processing system 100 will be described in order. Hereinafter, an explanation will be given by narrowing down the various functions possessed by each device to the functions related to the process of scanning a document, transforming the scanned image into a file, and then saving the file in the Storage Server 130.

[0048] <<Software Structure of MFP>>

[0049] The functional modules of the MFP 110 are roughly divided into a local functional module 410 and an additional functional module 420. The local functional module 410 is an application set in the MFP 110 as a standard application, and the additional functional module 420 is an application additionally installed in the MFP 110. The additional functional module 420 is a Java (registered trademark)-based application and can easily add functions to the MFP 110. Other additional applications not shown can be installed in the MFP 110.

[0050] The local functional module 410 has a scan execution unit 411 and a scanned image management unit 412. In addition, the additional functional module 420 has a display control unit 421, a scan control unit 422, a collaboration service request unit 423, and an image processing unit 424.

[0051] The display control unit 421 displays the user interface screen (UI screen) for receiving various user operations on the touch panel of the operation unit 220. These user operations include, for example, inputting login authentication information for accessing the MFP collaboration server 120; providing scan settings; providing a scan start instruction; setting filenames; and providing a file save instruction.

[0052] Based on user actions on the UI screen (e.g., pressing the "Start Scan" button), the scan control unit 422 provides the scan execution unit 411 with instructions for scan processing along with scan settings information. The scan execution unit 411, based on the instructions from the scan control unit 422, causes the scanner unit 222 to perform document reading operations via the scanner I / F 217 and generate scanned image data. The generated scanned image data is stored in the HDD 214 by the scanned image management unit 412. At this time, the scan control unit 422 is notified of information related to a scanned image identifier that uniquely represents the stored scanned image data. The scanned image identifier is a number, symbol, or letter used to uniquely identify the image scanned in the MFP 110. The scan control unit 422, for example, uses the aforementioned scanned image identifier to obtain the object scanned image data, which is then converted into a document, from the scanned image management unit 412. Then, the scan control unit 422 instructs the collaboration service request unit 423 to request the MFP collaboration server 120 for the processing required to convert the scanned image data into a document.

[0053] The collaboration service request unit 423 sends requests to the MFP collaboration server 120 for various processes and receives responses to these requests, etc. These processes include, for example, login authentication, scanned image analysis, and scanned image data transmission. Communication protocols such as REST and SOAP are used for sending and receiving data with the MFP collaboration server 120.

[0054] The image processing unit 424 generates an image for use on the UI screen displayed by the display control unit 421 by performing predetermined image processing on the scanned image data. Details of the predetermined image processing will be described later.

[0055] <<Software Architecture of Server Devices>>

[0056] First, the software structure of the MFP collaboration server 120 will be explained. The MFP collaboration server 120 includes a request control unit 431, an image processing unit 432, a storage server access unit 433, a data management unit 434, and a display control unit 435. The request control unit 431 is in standby mode, capable of receiving requests from external devices, and instructs the image processing unit 432, the storage server access unit 433, and the data management unit 434 to perform predetermined processing based on the received request content. The image processing unit 432 performs image analysis processing on the scanned image data sent from the MFP 110, including image analysis processing such as character region detection, character recognition, and similar document judgment, as well as image modification processing such as rotation and tilt correction. The storage server access unit 433 makes requests to the storage server 130. The cloud service exposes various interfaces for saving files to the storage server and retrieving saved files using protocols such as REST and SOAP. The storage server access unit 433 makes requests to the storage server 130 using the exposed interfaces. The data management unit 434 stores and manages user information and various setting data managed in the MFP collaboration server 120. The display control unit 435 receives requests from a web browser running on a PC or mobile terminal (neither shown) connected via the Internet and returns the screen structure information (HTML and CSS, etc.) required for screen display. Users can check registered user information and change scanning settings, etc., through the screen displayed on the web browser.

[0057] Next, the software structure of the storage server 130 will be described. The storage server 130 includes a request control unit 441, a file management unit 442, and a display control unit 443. The request control unit 441 is in standby mode, capable of receiving requests from external devices, and in this embodiment, instructs the file management unit 442 to save and read received files based on requests from the MFP collaboration server 120. Then, the request control unit 441 returns a response corresponding to the request to the MFP collaboration server 120. The display control unit 443 receives requests from web browsers running on PCs or mobile terminals (neither shown) connected via the Internet, and returns screen configuration information (HTML and CSS, etc.) required for screen display. Users can inspect and retrieve saved files, etc., via the screen displayed on the web browser.

[0058] <Processing flow of the entire image processing system>

[0059] Figure 5This is a sequence diagram of the processing flow between devices when scanning documents in an MFP 110, converting the obtained scanned images into files, and saving the files in a storage server. Figure 6 This is an example diagram showing the UI screen (hereinafter referred to as the "main screen") of the main menu displayed when the MFP 110 starts up. After scanning a document and converting the scanned image into a file, and installing the dedicated application required for using cloud storage services on the MFP 110, a "Scan and Save to Cloud Storage" button 601 is displayed on the main screen 600. Then, when the user presses the "Scan and Save to Cloud Storage" button 601 from the menu buttons displayed within the main screen 600, Figure 5 The sequence diagram shows a series of processes that begin. Below, according to... Figure 5 The sequence diagram illustrates the operations between devices in a time sequence. In the following description, the symbol "S" denotes a step.

[0060] MFP 110 displays a UI screen (hereinafter referred to as the "login screen") on the operation unit 220, on which information related to login authentication for accessing the MFP collaboration server 120 is entered (S501). Figure 7 An example of a login screen is shown. When the user enters the pre-registered user ID and password in the input fields 702 and 703 on the login screen 700 and presses the "Login" button 701, a login authentication request is sent to the MFP collaboration server 120 (S502).

[0061] Upon receiving a login authentication request, the MFP collaboration server 120 performs authentication using the user ID and password included in the request (S503). If the authentication result confirms that the user is a legitimate user, the MFP collaboration server 120 returns an access token to the MFP 110. Subsequently, the access token is sent along with various requests from the MFP 110 to the MFP collaboration server 120 to designate the logged-in user. In this embodiment, it is assumed that login to the MFP collaboration server 120 is completed simultaneously with login to the storage server 130. Therefore, the user pre-associates the user ID used for using the MFP collaboration service with the user ID used for using the storage service via a web browser on a PC (not shown) on the Internet. Thus, if login authentication to the MFP collaboration server 120 is successful, login authentication to the storage server 130 is also completed simultaneously, and the operation of logging into the storage server 130 can be omitted. Then, the MFP collaboration server 120 can handle storage service-related requests from users who have logged into the MFP collaboration server 120. Login authentication can typically be performed using well-known methods (such as basic authentication, digest authentication, and authentication using OAuth).

[0062] Upon successful login, the MFP 110 displays the UI screen used for scanning settings (hereinafter referred to as the "scan settings screen") on the operation unit 220 (S504). Figure 8 An example of a scan settings screen is shown. On the scan settings screen 800, there is a "Start Scan" button 801, a color settings bar 802, and a resolution settings bar 803. The "Start Scan" button 801 is a button used to give an instruction to begin scanning processing of a document placed on the document tray (in this embodiment, it is assumed to be a business form such as a quotation or invoice). In the color settings bar 802, the color mode for scanning is set. For example, the color settings bar 802 is designed so that the color mode can be specified from options such as full color and monochrome. In the resolution settings bar 803, the resolution for scanning is set. For example, the resolution settings bar 803 is designed so that the resolution can be specified from options such as 600 dpi and 1200 dpi. The color mode and resolution settings are merely exemplary, and all of these items are unnecessary, or other settings may exist besides these. Furthermore, the options related to color mode and resolution can be limited to settings required by the storage service. Detailed condition settings for scanning processing are provided by the logged-in user via a scan settings screen 800 such as this. After the user completes the scanning settings, places the document to be scanned on the document tray of the MFP 110 and presses the "Start Scan" button 801, scanning begins (S505). This generates image data obtained by computerizing the paper document. After the scan is complete, the MFP 110 sends the scanned image data, along with an analysis request for that image data, to the MFP collaboration server 120 (S506).

[0063] Upon receiving a request to analyze a scanned image, the MFP collaboration server 120 instructs the image processing unit 432 to perform image analysis processing (S507). At this time, the request control unit 431 returns a request ID that uniquely identifies the received analysis request to the MFP 110. Figure 9AAn example of a request ID is shown. On the other hand, the image processing unit 432, having received an instruction to perform analysis processing, analyzes the scanned image (S508). In this image analysis processing, firstly, processing is performed to detect character regions (text blocks) present within the scanned image. For character region detection, known methods can be applied, such as methods for extracting rectangular regions predicted as characters from an image binarized using a specific threshold. Next, optical character recognition (OCR) is performed on each character region detected by the character region detection processing. Then, processing is performed to determine whether the current scanned business form is similar to a past scanned business form (similar document determination processing). The determination of similar business forms uses configuration information indicating the location of each character region within the scanned image. Specifically, configuration information related to past scanned images and configuration information related to the current scanned image are compared, and it is determined whether the configurations of the character regions are similar. This is based on the fact that if the configurations of the character regions are similar, it can be predicted that the business forms were created using the same format. The configuration information related to past scanned images used for determining similar business forms is accumulated through the "business form learning processing," which will be explained later. The results obtained through image analysis processing are transmitted to the request control unit 431. In this embodiment, the similarity between business forms is determined solely based on the degree of similarity between the configurations of character regions. However, for example, the type of business form (quotation and invoice, etc.) can also be specified based on the OCR results, and similarity can be determined by considering the obtained type information.

[0064] During the aforementioned image analysis processing, MFP 110 periodically (e.g., every few hundred milliseconds to every few milliseconds) queries MFP collaboration server 120 regarding the processing status using the aforementioned request ID (S509-S509'). This query is repeated until a completion response from MFP collaboration server 120 is received regarding the image analysis processing (S510). Upon receiving a query regarding the processing status, MFP collaboration server 120 checks the progress of the image analysis processing corresponding to the request ID and returns a response indicating that processing is ongoing while the image analysis processing is not yet complete (see...). Figure 9B Upon completion of image analysis processing, the MFP collaboration server 120 returns a response indicating that the image analysis processing is complete (see [link]). Figure 9CIn the "status" field of the response, enter a string indicating the current processing status. Specifically, enter "processing" if processing is in progress on the MFP collaboration server 120, and enter "completed" if processing is complete. There are also cases where strings indicating other statuses are entered (such as "failed" if processing failed). Figure 9C As shown, the response upon completion of processing includes not only status information but also information related to the analysis results of the scanned images.

[0065] Upon receiving the processing completion response, MFP 110 requests the image analysis processing results from MFP collaboration server 120 using the URL representing the storage destination of the image analysis results included in the response (S511). In this case, the URLs are "ocrResultUrl" and "matchingResultUrl". Upon receiving this request, the request control unit 431 of MFP collaboration server 120 returns result information related to the image analysis processing.

[0066] Then, the MFP 110 uses the acquired result information to display a UI screen for setting the file name (hereinafter referred to as the "file name setting screen") (S512). Figure 10 An example of a filename setting screen is shown. When the user sets a filename and presses the send button 1007, the MFP 110 first sends a learning request, which includes information related to the user's input operations when setting the filename (input information at the time of setting), to the MFP collaboration server 120 (S513). Figure 11 This example shows a learning request. The learning content is specified in "learningContent," which contains a "rectInfoArray" related to the character region used in the filename and a "metadataArray" related to metadata. In "rectInfoArray," enter the coordinate information related to the character region used when setting the filename. In "metadataArray," enter information related to the character region corresponding to the string used in the same filename, as well as information related to the delimiters appended to the string. Figure 11 The example shows that the filename structure is as follows: it begins with a string of the character region "fileRegion0", followed by a separator, and ends with a string of the character region "fileRegion1", and these three are arranged in this order.

[0067] In the MFP cooperation server 120 that has received a learning request, the request control unit 431 instructs the image processing unit 432 to perform commercial form learning processing (S514). When receiving the instruction, the image processing unit 432 saves configuration information related to each character area existing in the scanned image and input information (file name structure information) at the time of file name setting included in the learning request received in S513 (S515).

[0068] After that, the MFP 110 sends a request for saving the scanned image together with the scanned image data and information such as the file name set when transforming into a file to the MFP cooperation server 120 (S516). In the MFP cooperation server 120 that has received the save request, the request control unit 431, in addition to starting the file generation processing, returns a response indicating that the save request has been normally received to the MFP 110. The MFP 110 that has received the response terminates the processing, and the state returns to the state of displaying the scan setting screen (S504). On the other hand, the MFP cooperation server 120 obtains information related to the file format from the pre-registered scan settings, and converts the scanned image into a file according to this file format (S517). At this time, the file name specified in the save request is attached to the generated file. The scanned image file thus generated is sent to the storage server and saved in the storage server (S518).

[0069] The above is the processing flow of the entire image processing system.

[0070] <Details of MFP's processing>

[0071] Subsequently, by focusing on the operations in the MFP 110, the processing in the case of using the storage service in the above image processing system is described in detail. Figure 12 It is a flowchart showing the processing flow in the MFP 110. This series of processing is implemented by the CPU 211 executing the control program stored in the HDD 214 in the control unit 210, and starts in response to pressing the "Scan and Save in Cloud Storage" button 601 on the main screen 600 described above. The following gives a detailed description along Figure 12 the shown flowchart. The symbol "S" above each processing means step.

[0072] In S1201, the display control unit 421 displays the above-mentioned login screen 700. When the user name and password are respectively input to the input fields 702 and 703 on the login screen 700 and it is detected that the "Login" button 701 has been pressed, the processing proceeds to S1202.

[0073] In S1202, the collaboration service request unit 423 sends a login authentication request to the MFP collaboration server 120. If the login authentication check in the MFP collaboration server 120 confirms that the user is a real user, the collaboration service request unit 423 receives an access token from the MFP collaboration server 120.

[0074] In S1203, upon receiving the access token in MFP 110, the display control unit 421 displays the previously described scan settings screen 800 on the operation unit 220. When a logged-in user places a document on the document tray and the "Start Scan" button 801 is pressed, the display control unit 421 notifies the scan control unit 422 of this situation.

[0075] In S1204, the scan control unit 422, having received the aforementioned notification, instructs the scan execution unit 411 to perform scan processing. Upon receiving the instruction, the scan execution unit 411 scans the document placed on the document tray. In this process description, it is assumed that a business form (more specifically, a quotation) is used as an example of the scanned object. Thus, the following is obtained: Figure 13 The scanned image of the quotation shown is displayed. The scanned image data generated by the scan is stored in the scanned image management unit 412, and the display control unit 421 is notified that an identifier for the scanned image can be specified. At this time, a message screen (not shown) indicating that the scan is in progress can also be displayed on the operation unit 220.

[0076] Next, in S1205, the collaboration service request unit 423 acquires scanned image data via the scan control unit 422 and sends the scanned image data along with an analysis request to the MFP collaboration server 120. In the MFP collaboration server 120, based on the analysis request, the received scanned image data undergoes the image analysis processing described above. At this time, the request control unit 431 of the MFP collaboration server 120 returns a request ID.

[0077] In S1206, the collaboration service request unit 423, having received the request ID, sends a request to obtain the processing status along with the request ID to the MFP collaboration server 120. Based on this request, the MFP collaboration server 120 returns a response to the MFP 110 corresponding to the progress status of the image analysis processing specified by the request ID.

[0078] In S1207, if the received response indicates "processing" or "processing in progress," the collaboration service request unit 423 returns to S1206. Then, after waiting for a predetermined time, the collaboration service request unit 423 sends a request to the MFP collaboration server 120 again to obtain the processing status. If the received response indicates "completed" or "processing completed," processing proceeds to S1208. Here, refer to the previously described... Figure 9C This section describes information related to the image analysis results included in the response upon completion of processing in this embodiment. As information related to the image analysis results, there are three categories: "ocrResultUrl," "matchingResultUrl," and "formKeys." The "ocrResultUrl" information relates to the URL used to obtain the result of the OCR processing in the image analysis process. The "matchingResultUrl" information relates to the URL used to obtain the result of the similar business form judgment processing in the image analysis process. The "formKeys" information relates to the setting values ​​for setting attributes (such as filename, metadata, and folder path) of the scanned image, and is pre-set in the MFP collaboration server 120. The "formKeys" contains the following information.

[0079] • key: A value that uniquely represents the setting applied to the scanned image.

[0080] • keyType: A value indicating the type of the key's set value.

[0081] • value: Sets the initial value of the value (e.g., "scan" representing the scanned image).

[0082] • type: Indicates the type of value input into the setting (e.g., "string" for strings, "number" for numbers, and "date" for dates).

[0083] • displayName; The name to display when the setup screen is shown on the MFP's touch panel.

[0084] • required: Indicates whether the input value is required.

[0085] • multiSelect: Indicates whether to use strings from multiple character ranges in the filename value.

[0086] • separator: The separator used to join strings that use multiple character ranges.

[0087] • autoInput: A value indicating whether to automatically input the initial filename.

[0088] exist Figure 9C The text shows "formKeys" when a filename is set for a scanned image. This is achieved by using... Figure 9C Using this case as an example, we can illustrate how the values ​​of "value", "displayName", "required", and "multiSelect" affect the filename setting screen displayed in the following S1209. Figure 10 Currently, the content of "value" is empty. In this case, the initial value is not displayed in the filename input field 1001 on the filename setting screen 1000. Furthermore, "displayName" is "filename". In this case, compared to... Figure 10 Similar to the filename setting screen 1000, the string "Filename" representing the setting title or label is displayed at the top of the screen (in this example, as empty characters on a colored background). Furthermore, "required" is set to "true". In this case, the "Send" button 1007 can no longer be pressed if no string is entered in the filename input field 1001. Additionally, "multiSelect" is set to "true". In this case, multiple character regions detected from the scanned image can be selected, allowing the setting of filenames that concatenate strings corresponding to each character region. The above-mentioned content included in "formKeys" is set by the user via the filename condition setting screen displayed by the display control unit 435 of the MFP collaboration server 120. Figure 14 An example of a filename condition setting screen is shown. Checkbox 1401 within the filename condition setting screen 1400 is used by the user to select whether to automatically enter the filename using information related to the character regions previously used when setting filenames for scanned images. In the setting bar 1402, when multiple strings corresponding to multiple character regions are used for the filename, the separator inserted between strings is set from the options (underscores and hyphens, etc.) displayed in the drop-down list. Here, underscore is selected. In the filename setting screen ( Figure 10 When the file name condition setting screen 1400 is initially displayed, the setting bar 1403 is pre-entered into input mode, and a string (initial file name) is set as a candidate file name presented to the user. The content set on this file name condition setting screen 1400 is stored in the data management unit 434 and is referenced in the response when the creation process is completed.

[0089] In S1208, the collaborative service request unit 423 accesses the URL included in the response upon completion of processing and obtains the image analysis results. Figure 9C In the example, information is stored in "ocrResultUrl" and "matchingResultUrl", so the collaboration service request unit 423 accesses these two URLs and obtains the image analysis results. Table 1 below shows the results from... Figure 13 The coordinates of the character regions detected in the scanned image and the character recognition results (recognized strings) for each character region.

[0090] Table 1

[0091]

[0092] Then, Figure 15 This shows the result obtained by accessing "ocrResultUrl". Figure 13 The results of character recognition for scanned images are presented. Here, "imageWidth" represents the number of pixels in the X direction (horizontal direction) of the scanned image of the analyzed object, and "imageHeight" represents the number of pixels in the Y direction (vertical direction) of the scanned image of the analyzed object. "regions" includes coordinate information ("rect") related to the character regions detected from the scanned image and string information ("Text") corresponding to the character regions. "rect" includes information specifying the detected character regions: "x" represents the X coordinate of the top-left corner of the character region, "y" represents the Y coordinate of the top-left corner of the character region, "width" represents the number of pixels in the X (width) direction of the character region, and "height" represents the number of pixels in the Y (height) direction of the character region. Then, "text" represents information related to the string obtained by performing OCR processing on the character region specified by "rect". The information "rect" and "text" are included to correspond to all character regions detected within the scanned image of the analyzed object. However, in... Figure 15 In the text, a portion of these character regions has been omitted.

[0093] Then, Figure 16 This shows the results obtained by accessing "matchingResultUrl". Figure 13The results of the similarity assessment of scanned images for business forms are as follows. In "matched", enter "true" or "false" to indicate whether a similar scanned image was found in past scanned images through the aforementioned similarity assessment process. A value of "true" indicates that the current scanned business form is similar to a previously scanned business form, and "false" indicates that the current scanned business form is not similar to a previously scanned business form. In "formID", if a similar business form is found, enter the value (ID value) that uniquely represents a scanned image of a past similar business form. On the other hand, if no similar business form is found, enter a newly created ID value that uniquely represents a scanned image of the business form targeted in this scan. Then, the newly created ID value is used when learning the input information (structural information related to the filename) when setting the filename of scanned images for business forms for which no similar business forms were found in the past. In "matchingScore", if a similar business form exists, enter a value representing the consistency rate. In this embodiment, the similarity level is represented by a real value between 0 and 1, indicating the consistency rate between configuration information related to character regions in past scanned images and configuration information related to character regions in the current scanned image. This means that the larger the value, the more similar the two are. In "rectInfoArray", information representing the character regions in the current scanned image is input, corresponding to the character regions used when setting the filenames of scanned images of similar past business forms.

[0094] exist Figure 16 In the example case, for those obtained by scanning a quotation. Figure 13 The scanned images are used to set filenames using two strings ("quotation" and "Shinagawa Co., Ltd."), and the system learns from the input information. Then, quotation files created in the same format are scanned and generated. Figure 17 The scanned image shown, and as a result of performing a similar business form judgment process on the scanned image, is judged to be similar to... Figure 13 The scanned images are similar. Then, Figure 16 The example shows that it will be compared with the one for targeting Figure 13 Information about the character region used when setting the filename of the scanned image is stored as a state related to the character region of the auto-input object. The following explains the process until the filename is obtained. Figure 16 The processing continues until the result of the similar business form judgment is shown.

[0095] As a premise, it is assumed that the above will be based on Figure 11The learning results of the learning request are stored in the data management unit 434. Then, in the similar business form judgment processing, by using coordinate information related to the character regions of "quotation" and "Shinagawa Co., Ltd." used in the previous file name settings, a character region that partially overlaps with the character region represented by the coordinate information is specified in the character region included in the current scanned image. Then, the coordinate information related to the partially overlapped character region and the string of the character region are stored in the "text" of "rectInfoArray". In "key", a value uniquely representing the character region used for automatic input of the file name is entered. In "region", coordinate information related to the character region uniquely represented by "key" and the string obtained by OCR processing of the character region are stored. Then, in "rect", information for specifying the character region uniquely represented by "key" is included. In this case, "x" represents the X coordinate of the upper left corner of the character region, "y" represents the Y coordinate of the upper left corner of the character region, "width" represents the number of pixels in the X (width) direction of the character region, and "height" represents the number of pixels in the Y (height) direction of the character region. Next, enter information related to the string obtained by OCR processing of the character region specified by "rect" in "text". In "metadataArray", store the order of the character regions used when automatically entering the filename and information indicating where the delimiter is inserted. If attribute information such as metadata is set in addition to the filename, add the required information to "rectInfoArray" and "metadataArray". In "key", store a value that uniquely represents the setting value set to the scanned image. In "keyType", store a value indicating the type of the setting value for "key". In the case of a filename, enter "filename" in both "key" and "filename" in "keyType". In "value", store the character region used for the value of "key" and information related to the delimiter. Figure 16 In the example, the string of the character region with the key "fileRegion0" in "rectInfoArray" is the first, the delimiter is the second, and the string of the character region with the key "fileRegion1" is the third. Then, in this order, these three are automatically entered into the filename input field 1001 and displayed as the initial filename.

[0096] In S1209, the display control unit 421 displays the aforementioned filename setting screen 1000 on the operation unit 220, allowing the user to set filenames for the scanned images. Details of this filename setting process will be explained later.

[0097] In S1210, the collaboration service request unit 423 determines whether automatic filename input is enabled and determines the next step by referring to the response to the request for processing status in S1206. Specifically, the collaboration service request unit 423 refers to the response described above. Figure 9C The response shown upon completion of the processing includes the value of "autoInput". If "true" indicates automatic input, the processing proceeds to S1211. Conversely, if "false" indicates no automatic input, the processing proceeds to S1214.

[0098] In S1211, the collaborative service request unit 423 determines whether a similar scanned image exists in the past based on the image analysis results obtained in S1208. Specifically, the collaborative service request unit 423 refers to the previously described... Figure 16 The result of the similar business form judgment process shown includes the value of "matched". If "true" indicates that a similar scanned image exists in the past scanned images, the process proceeds to S1212. On the other hand, if "false" indicates that no similar scanned image exists, the process proceeds to S1214.

[0099] In S1212, the collaboration service request unit 423 determines whether the user has changed the initial filename automatically entered in the filename input field 1001 during the filename setting process in S1209. Changes in this case may include, for example, changing the character range used in the current filename to a different character range than the initial filename; adding or deleting separators; etc. Then, by... Figure 16 The judgment is made by comparing the value of "metadataArray" included in the result of the similar business form judgment process with the information entered in the filename input field 1001 after setting the filename of the scanned image. As a result of this comparison, if there is a difference, it is determined that the initial filename has been changed and the process proceeds to S1213; if the two are the same, it is determined that the initial filename has not been changed and the process proceeds to S1214. Regarding the starting string of the filename automatically entered through the "value" of "formKeys", even if the user deletes the string, the process can proceed to S1214 without assuming that the input information has been changed.

[0100] In S1213, the collaboration service request unit 423 will request the learning of the input information during setup (see above). Figure 11 The data is sent to the MFP collaboration server 120. The input information during this setup indicates the filename structure set for the scanned image. The "formId" included in the learning request in this case is the value of the "formId" included in the similar business form judgment processing result obtained in S1208, which is the "formId" in the learning request. Figure 16 In the example, the input "aaaaaaaa-ffff-49ab-acf8-55558888eeee" is used as the value of "formId", and therefore this value is stored in the learning request. Furthermore, the "learningContent" stores the input information used when setting the filename. First, in "rectInfoArray", coordinate information related to the character region to which the string used in the filename belongs is stored, corresponding to the number of character regions. Then, in "metadataArray", information related to the character region and separator used in the filename setting is stored. Referring to Table 1 above, to illustrate the user's actions at this point, first, the user touches "Quotation Form" with number 1 in the preview area 1002 on the filename setting screen 1000. Next, the user touches "Shinagawa Co., Ltd." with number 8, and finally presses the "Send" button 1007. The MFP collaboration server 120, receiving the learning request, accumulates the input information based on user actions such as these, and uses the accumulated input information in the next and subsequent image analysis processing. After sending the learning request, if a response indicating that the learning request has been processed normally is received from the MFP collaboration server 120, the process proceeds to S1214.

[0101] In S1214, the collaboration service request unit 423 associates the scanned image data of the processing object with the filename set in S1209 and sends both to the MFP collaboration server 120. Once this transmission is complete, the processing ends. In this embodiment, the transmission of the scanned image data and its filename is performed last, but it is also possible to transmit both to the MFP collaboration server 120 in parallel at the point in time when the scanned image data is saved using the scanned image management unit 412, after the scanning process is completed.

[0102] The above describes the processing flow when using a storage service focused on operations within the MFP 110. In the above description, the following scenario is assumed: setting a filename when converting a scanned image obtained by scanning a quotation into a file. However, this embodiment is not limited to this, and can also be applied to scenarios where filenames are set when converting multiple scanned images obtained by scanning multiple quotations into files. In this case, a page-turning button can be provided within the filename setting screen 1000, allowing filenames to be set from the character area of ​​the scanned image on a second or third page.

[0103] (Filename settings processing)

[0104] Then, along Figure 18A , 18B The flowcharts shown in 19A and 19B illustrate in detail the file name setting process performed by the control unit 421 in S1209 described above.

[0105] In S1801, a filename (initial filename) is generated and displayed in its initial state in the filename input field 1001 on the filename setting screen 1000 described above. The initial filename is generated through the following process. First, the display control unit 421 refers to the response when the processing described above is completed, and sets a value to the beginning of the filename if a value is entered in the "value" field of "formKeys". For example, if the string "scan" is entered as the "value" field of "formKeys", the display control unit 421 sets "scan" to the beginning of the initial filename. Next, based on the similar business form judgment result obtained by accessing "mathingResultUrl", the display control unit 421 sets a predetermined string. That is, if the value of "matched" is "true" and the "matchingScore" is a value greater than or equal to a predetermined value (e.g., 0.6 or greater), the display control unit 421 sets the string according to the filename structure specified by the "value" field of "metadataArray". Specifically, the display control unit 421 sets the string of the character area specified by "rectInfoArray" according to the order specified in the "value" field. In the example of similar business form judgment results, the value "matched" is "true" and "matchingScore" is "0.74...", so this is a value greater than or equal to the predetermined value. Then, in the "value" field of "metadataArray", "fileRegion0" is specified as the first, "separator" as the next, and "fileRegion1" as the last. As a result, "quotation" as the value of "text" for "fileRegion0", "_ (underscore)" as the value of "separator" included in the processing completion response, and "Shimomaruko Co., Ltd." as the value of "text" for "fileRegion1" are connected in this order. In this way, "quotation_Shimomaruko Co., Ltd." is generated as the initial filename. If a value (e.g., "scan_") is entered in the "value" field of "formKeys", "scan_quotation_Shimomaruko Co., Ltd." is obtained by adding "scan_" to "quotation_Shimomaruko Co., Ltd." as the initial filename. In the similar business form judgment results, if the value of "matched" is "false" and "matchedScore" is less than the predetermined value, and a string is entered in the "value" field of "formKeys", the string will be treated as the original file name.Furthermore, if no string is entered in the "value" field of "formKeys", the initial filename is not displayed (empty field). In this embodiment, the values ​​of "matched" and "matchedScore" are used (i.e., based on "meta"). data The order of the "values" in the "Array" (information specified in "rectInfoArray") is used as a condition for generating the initial filename. However, the conditions are not limited to this, and one of these results can also be used as a condition. For example, the initial filename can also be generated by using only the value of "matched" as "true" as a condition, or by using only the value representing the degree of similarity as greater than or equal to a threshold as a condition.

[0106] In S1802, it is determined to what extent the scanned image is scaled down (hereinafter referred to as the initial magnification) when it is initially displayed in the preview area 1002 on the filename setting screen 1000. To determine the initial magnification, the character recognition results obtained by accessing "ocrResultUrl" are used. First, the leftmost and rightmost character areas are found in all detected character areas within the scanned image. In this case, the leftmost character area is the character area with the minimum x-coordinate value among all character areas. Furthermore, the rightmost character area is the character area with the largest sum of the x-coordinate value and the width of the character area among all character areas. The initial magnification is determined by the magnification that makes the left side of the thus determined leftmost character area coincide with the left side of the preview area 1002, and the right side of the rightmost character area coincide with the right side of the preview area 1002. Since it is possible that the user may not be able to easily select (press) the image when the ends of the determined character areas are completely aligned with the ends of the preview area, a predetermined margin can also be given to determine the initial magnification. For example, an initial magnification is determined such that a margin corresponding to 10 pixels is ensured between the left edge of the preview area 1002 and the left edge of the character area displayed at the leftmost position within the preview area 1002, and between the right edge of the preview area 1002 and the right edge of the character area displayed at the rightmost position. Furthermore, when UI components such as various operation buttons are displayed overlappingly on the preview area 1002, an initial magnification can also be determined such that the character areas do not overlap with the UI components. Specifically, an initial magnification can also be determined such that after subtracting the width of the UI components from the width of the preview area 1002, both the leftmost and rightmost character areas are included within the width of the remaining area. As described above... Figure 10In the example of the filename setting screen 1000, three operation buttons (zoom in button 1004, zoom out button 1005, and restore button 1006) are present in the preview area 1002. Therefore, an initial magnification is determined so that the character area does not overlap with these buttons. Furthermore, there are cases where header information related to printing processing may be printed near the top, bottom, left, and right edges of the scanned object's business form, and the likelihood of using a string corresponding to such header information as part of the filename is low. As a result, when the aforementioned leftmost and rightmost character areas are found, character areas existing within a predetermined distance (e.g., 50 pixels) can be excluded from the top, bottom, left, and right edges of the scanned image.

[0107] In S1803, it is determined which portion of the scanned image, after being magnified / reduced at the initial magnification determined in S1802, will be displayed in its initial state within the preview area 1002 on the filename setting screen 1000. In the scanned image after being magnified / reduced using the magnification determined in S1802, the widths of the leftmost and rightmost character areas are approximately the same as the width of the preview area 1002. As a result, firstly, in the horizontal direction, the portion displayed in its initial state (hereinafter referred to as the "initial display portion") is determined such that the leftmost and rightmost character areas are exactly included in the preview area 1002; for example, the left side of the leftmost character area coincides with the left side of the preview area 1002. Then, the initial display portion in the vertical direction is determined such that the uppermost character area is located near the upper end of the preview area 1002. At this point, since the margins were considered when determining the initial magnification, it is sufficient to determine the initial display portion by also considering the margins. For example, character regions existing within a predetermined distance (e.g., 50 pixels) from the top of the self-scanned image can be excluded from the display object, and an initial display portion can be determined such that the character region located at the topmost position among the character regions other than the excluded character region is displayed at the top of the preview area 1002. Figure 20 It shows that it is aimed at Figure 13 The image shows the initial display portion determined by the scanned image. Figure 20The wireframe in the preview area 1002 represents the range of the scanned image displayed in its initial state. As a result of determining the initial display portion as described above, there are cases where the character area of ​​a string is not displayed in the preview area 1002, and this character area is used for the initial filename under the condition that a similar business form exists and the initial filename is automatically entered in the filename input field 1001. In this case, it is sufficient to determine the initial display portion such that the character area of ​​the string used for the initial filename is displayed in the center of the preview area 1002. At this time, it is also possible to highlight the string constituting the initial filename automatically entered in the filename input field 1001 and the character area of ​​the string used for that initial filename (for example, by changing the color of both). Figure 21 Is Figure 13 This is an example of a filename setting screen 1000 where the string "Kawasaki Corporation" is automatically entered as the initial filename in a scanned image. Here, the character area and the string "Kawasaki Corporation" entered in the filename input field 1001 are displayed, such that the character area corresponding to the string "Kawasaki Corporation" is centered in the preview area 1002, and the color of this character area is further changed. At this time, if the character size in the highlighted character area is small, the initial magnification can be adjusted to achieve an appropriate character size. The adjusted initial magnification can be obtained, for example, based on the size of the touch panel of the operation unit 220 and the range displayed in the preview area 1002 in the initial state. Specifically, it is sufficient to calculate the character size point based on the height of the character area of ​​the string used in the initial filename, and adjust the initial magnification of the preview so that the calculated point becomes greater than or equal to a threshold (greater than or equal to a predetermined size).

[0108] In S1804, using the processing results of S1801 to S1803, the previously described filename setting screen 1000 is displayed on the operation unit 220. The user then uses... Figure 10 The filename setting screen 1000 shown is used to set the filename of the scanned images saved in the storage server 130. This will be explained in detail below.

[0109] 1) File name input field

[0110] When the user touches the file name input field 1001, the display of the touch panel of the operation unit 220 switches to a soft keyboard, allowing the user to input any characters and edit automatically entered strings. For example, as described above... Figure 21 As in the example, when the user touches the string portion of the filename input field 1001 that automatically inputs "Kawasaki Corporation", it displays... Figure 22 The soft keyboard 2200 is shown. At this time, as... Figure 22As shown, the soft keyboard 2200 is displayed in a state where the touched string is entered in the input area 2201. Thus, the user can modify the touched string "Kawasaki Corporation" by operating the soft keyboard 2200. Furthermore, when touching... Figure 10 When the filename input field 1001 is in a state where no string has been entered, the soft keyboard 2200 is displayed in the input area 2201 when no content has been entered. The user can enter any string in the filename input field 1001 by operating the soft keyboard 2200.

[0111] 2) Preview area

[0112] In preview area 1002, as described above, a specific portion of the scanned image to which the zoom processing is applied is displayed. Then, when the user touches and selects any character area within preview area 1002, the string representing the selected character area is added to filename input field 1001. At this time, the selected character area can be highlighted by underlining the selected area, surrounding it with a box, or coloring it, making it clear that the selected character area has been selected. Furthermore, the display portion and magnification can be changed to center the selected character area. Additionally, when multiple character areas are selected, they can be colored differently to differentiate their appearance. Then, when the user touches the previously selected character area again, the selection is deselected (i.e., the lines or colors attached to the character area are removed and the original state is returned), and the string corresponding to that character area in filename input field 1001 is also deleted. In this embodiment, when no character area is selected, boxes representing each character area are not displayed in preview area 1002. However, boxes representing each character area can also be pre-displayed, allowing users to identify the parts they can touch and change the color of those parts when selected, etc. Furthermore, users can toggle the display of boxes, allowing them to identify each character area via buttons, etc. Then, users can move the image portion displayed by sliding within the preview area 1002.

[0113] 3) Various operation buttons

[0114] The "×" button 1003 is a delete button used to delete the end of the string entered in the filename input field 1001. The "+" button 1004 is a zoom button used to zoom in on the image currently displayed in the preview area 1002, and is used when it is desired to display each character area in zoomed-in mode. The "-" button 1005 is a zoom-out button used to zoom out on the image currently displayed in the preview area 1002, and is used when it is desired to display a wider area. When zooming in or out, the display position is adjusted so that the coordinates of the center in the preview area 1002 are the same before and after zooming in or out. The "U-shaped arrow" button 1006 is a restore button used to return the preview area 1002 to its initial display state if the display position has been moved by a sliding operation, or if the display magnification has been changed by pressing the zoom in button 1004 or the zoom out button 1005. The "Send" button 1007 is used to convert the scanned image into a file with the filename set on the filename setting screen 1000 and save the file in the storage server 130. In response to pressing the "Send" button 1007, the scanned image data along with information related to the set filename is sent to the MFP collaboration server 120.

[0115] In S1805, an operation guide is displayed on the filename setting screen 1000 in the initial display state. This guide allows first-time users to easily understand how to set the filename by touching and selecting the character area within the preview area 1002. Figure 23 Shown in Figure 10 The status of the operation guide message 2301 is displayed in an overlapping manner at the top of the preview area 1002 in the filename setting screen 1000 shown. Such operation guides are unnecessary for users who already know the operation method; therefore, at the time when certain user operations such as touch and swipe operations are detected, the operation guide is set not to be displayed. Furthermore, in many cases, there are characters or numbers important to the file, such as document titles, company names, business form numbers, and dates, at the top of a general document. As a result, the image portion displayed in the preview area 1002 can be moved down according to the attributes of the strings, so that the character areas of these important strings do not overlap with the operation guide. Furthermore, it is sufficient to determine the display position of the operation guide so that it does not overlap with or is not too close to the character area, in cases where the operation guide overlaps with one of the character areas, or when the operation guide is included within a predetermined distance from the character area. As described above, by flexibly determining the display position of the operation guide instead of a fixed display position, users can more easily select the character area.

[0116] S1806 is a process used to monitor the presence or absence of user operations. The presence or absence of user operations is checked at predetermined intervals, and if a certain user operation is detected, the process proceeds to S1807. Then, in S1807, as described above, the operation guidance is set not to be displayed.

[0117] S1808 to S18015 are determination processes used to determine the next step based on the content of the detected user operation. If the process corresponding to the detected user operation is completed, the process returns to S1806 and waits for the detection of the next user operation.

[0118] In S1808, it is determined whether the detected user operation is a touch operation on one of the character areas within the preview area 1002. Specifically, firstly, the coordinates of the touched position are transformed in association with the coordinates of the scanned image. Then, the transformed coordinates are included in the character recognition result described above (see...). Figure 15 If the user's action is within one of the character regions specified in "regions", it is determined that the user's action is a touch of a character region. If the determination result indicates a touch of a character region, the process proceeds to S1816; otherwise, the process proceeds to S1809. In S1816, if the user's action is a touch of a character region, it is determined whether the touched character region has been selected. If the touched character region is not yet selected, the process proceeds to S1817; otherwise, the process proceeds to S1818.

[0119] In S1817, the string of the touched character region is added to the filename input field 1001, and the character region is highlighted to indicate that it has been selected. At this time, if no other string is entered in the filename input field 1001, the string is added as is. Then, the fact that the character region is used for filename setting is stored as input information during filename setting. On the other hand, if the string of another character region has already been entered in the filename input field 1001, the string of the touched character region is added after adding a separator. In this case, in addition to the fact that the character region is used for filename setting, the fact that separators (such as "first character region," "separator," and "second character region") are inserted between strings is also stored. Figure 24This is a diagram showing the filename setting screen 1000 when the character area selected first is recognized as "quotation" and the character area selected later is recognized as "Shimo Maruko Co., Ltd." In the filename input field 1001, two strings corresponding to the order of the touched character areas are entered, and an underscore is entered as a separator between the two strings.

[0120] In S1818, the string representing the touched character area is deleted from the filename input field 1001, the highlight indicating selection of that character area is deselected, and the state returns to the normal display state indicating no character area is selected. Furthermore, if a separator was appended to the string, that separator is also deleted. For example, suppose the filename structure used to generate the initial filename is ["first character area", "separator", "second character area"]. In this case, if the user touches the character area of ​​the "first character area", the separator is also deleted along with the string of the first character area, and only the string of the "second character area" remains in the filename input field 1001.

[0121] In S1809, if the detected user operation is not a touch on the character area, it is determined whether the detected user operation is a swipe operation within the preview area 1002. If a swipe operation is detected, the process proceeds to S1819; otherwise, the process proceeds to S1810.

[0122] In S1819, processing is performed to move the portion of the image displayed in the preview area 1002 along the direction of the swipe operation. Specifically, the portion of the image displayed in the preview area 1002 is updated based on the amount of movement of the user's finger on the touch panel. The processing can also be limited to detecting touch operations in areas other than the character area within the preview area 1002. Furthermore, even if the user operation is a touch operation on the character area, it can be treated as a swipe operation based on the amount of subsequent finger movement by the user, instead of treating it as a touch on the character area.

[0123] In S1810, it is determined whether the detected user operation is the pressing of the zoom-in button 1004 or the zoom-out button 1005. If the zoom-in button 1004 or the zoom-out button 1005 is pressed, the process proceeds to S1820; otherwise, the process proceeds to S1811.

[0124] In S1820, processing is performed to magnify or reduce the portion of the image currently displayed in the preview area 1002. Specifically, when the magnification button 1004 is pressed, processing is performed to increase the magnification of the scanned image by a predetermined amount, and when the magnification button 1005 is pressed, processing is performed to reduce the magnification of the scanned image by a predetermined amount. Here, the predetermined amount is arbitrary, and for example, approximately 10% based on the current magnification. At this time, if during the magnification or reduction by the predetermined amount, there is an initial magnification or a magnification where the width / height of the scanned image and the width / height of the preview area 1002 are consistent with each other, the magnification or reduction can be stopped at that magnification. Control is performed so that the image portion is not reduced at a magnification (minimum magnification) or less than that where the entire scanned image is included in the preview area 1002, and the image portion is not magnified at a magnification (maximum magnification) or greater than that where the minimum character area becomes a predetermined size.

[0125] In S1811, it is determined whether the detected user operation is the pressing of the restore button 1006. If the restore button 1006 is pressed, the process proceeds to S1821; otherwise, the process proceeds to S1812.

[0126] In S1821, a process is performed to return the preview area 1002 to its initial display state. Specifically, the following process is performed: the magnification of the scanned image displayed in the preview area 1002 is returned to the initial magnification determined in S1802, and the display portion in the preview area 1002 is further returned to the initial display portion determined in S1803. At this time, since the initial magnification and initial display portion have been adjusted, it is desirable to return the magnification and display portion to the adjusted initial magnification and initial display portion.

[0127] In S1812, it is determined whether the detected user operation is a touch operation on the string entered in the filename input field 1001. If a touch operation on the blank area following the string entered in the filename input field 1001 is detected, it is sufficient to treat the touch operation as a touch operation on the end of the string. If a touch operation on the string entered in the filename input field 1001 is detected, the process proceeds to S1822; otherwise, the process proceeds to S1813.

[0128] In S1822, the soft keyboard for editing is displayed on the operation unit 220, indicating that a string corresponding to a touch operation has been entered in the input area 2201. Here, it is assumed that the filename setting screen 1000 is in... Figure 24The state shown. In this state, when a touch operation on the string "quotation" in the file name input field 1001 is detected, the soft keyboard 2200 in the state of inputting the string "quotation" in the input area 2201 of the soft keyboard is displayed. In addition, when a touch operation on the string "Shimamaruko Co., Ltd." or a touch operation on the blank area after this string is detected, the soft keyboard 2200 in the state of inputting the string "_Shimamaruko Co., Ltd." as the ending string in the input area 2201 is displayed. The operation of the soft keyboard 2200 is the same as that of a general keyboard, so the description of the soft keyboard 2200 is omitted. The subsequent S1823 is a process for monitoring the presence or absence of user operations. It checks at a predetermined interval whether the OK button 2202 in the soft keyboard 2200 has been pressed, and when the pressing of the OK button 2202 is detected, the process proceeds to S1824. When a key other than the OK button is pressed, the string displayed in the input area 2201 is edited according to the pressed key. For example, the user can modify the string misrecognized in the character recognition process by using the soft keyboard 2200.

[0129] In S1824, it is judged whether a change for deleting the delimiter has been made. In this embodiment, this judgment is made only when the string touched in S1812 is displayed together with the delimiter in S1822, and the string displayed without a delimiter is treated as a string for which no delimiter deletion has been made. That is, when the string is displayed in a state where a delimiter is attached to the beginning of the string, it is judged whether a string edit for deleting the delimiter has been made based on whether there is a delimiter at the beginning of the string at the time point when the OK button 2202 is pressed. For example, when a touch operation on the string "Shimamaruko Co., Ltd." is performed in S1812, as described above, the string in the state of "_Shimamaruko Co., Ltd." with a delimiter attached to the beginning is displayed in the input area 2201 of the soft keyboard 2200. Then, it is judged whether the delimiter has been deleted by checking whether the delimiter remains at the beginning. In addition, when a touch operation on the string "quotation" is performed in S1812, as described above, "quotation" is displayed in the input area 2201, so in this case, the string is treated as a string for which no change for deleting the delimiter has been made.

[0130] In the example above, an explanation was given regarding the display of a string with a separator appended to the beginning in the input area 2201, assuming a separator precedes the string touched in S1812. However, this is not a limitation. For example, if a separator follows the string touched in S1812, the same judgment can be made if the configuration is such that the touched string is displayed in the input area 2201 with a separator appended to the end. In other words, the following situation was explained: According to Figure 24 The example touch operation on the string "quotation" is configured so that the string is displayed in the input area 2201 of the soft keyboard 2200 with a delimiter appended to the end of the string, such as "quotation_". In this case, it is sufficient to determine whether the delimiter has been deleted by checking whether the delimiter is retained at the end in step S1824.

[0131] If it is determined in S1824 that a separator has been deleted, the process proceeds to S1825. Then, in S1825, the fact that the separator has been deleted from the filename is stored. For example, assume that the filename structure used to generate the initial filename is ["first character area", "separator", "second character area"]. If the user touches the string corresponding to "second character area" in the filename input field 1001 and deletes the separator using the soft keyboard 2200, the changed filename structure will be ["first character area", "second character area"]. Then, information related to this changed filename structure is stored. In the subsequent S1826, it is determined whether a change to add a separator to the filename has been made. If it is determined that a separator has been added, the process proceeds to S1827. Then, in S1827, the fact that a separator has been added to the filename is stored. For example, assume that the filename structure used to generate the initial filename is ["first character area", "second character area"]. At this point, if the user touches the string corresponding to "second character area" in the filename input field 1001 and adds a delimiter to the beginning of the string using the soft keyboard 2200, the changed filename structure will be ["first character area", "delimiter", "second character area"]. Then, information related to this changed filename structure is stored.

[0132] Then, in S1828, after the display screen of the operation unit 220 returns from the soft keyboard 2200 to the file name setting screen 1000, the file name is modified to the changed content. At this time, the display portion of the scanned image can also be changed so that the character area corresponding to the modified string is located in the center of the preview area 1002. Even if a part of the string used in the file name is modified, no new information related to the file name structure is stored if the file name structure remains unchanged. However, if a modification is made to delete all characters included in a specific string, the file name structure changes. As a result, in the same case (although this is omitted in the flowchart), structural information related to the changed file name is stored. For example, suppose the file name structure used to generate the initial file name is ["first character area", "separator", "second character area"]. At this time, suppose the user touches the string corresponding to the "second character area" in the file name input field 1001 and deletes all characters using the soft keyboard 2200. In this case, the unnecessary separator is also deleted, and the changed file name structure changes to ["first character area"], so information related to the changed file name structure is stored.

[0133] In S1813, it is determined whether the detected user operation is a touch operation on the empty filename input field 1001, which is in a state where no string has been entered. If it is a touch operation on the empty filename input field 1001, the process proceeds to S1829; otherwise, the process proceeds to S1814.

[0134] In S1829, the string that is currently in a state where no content has been entered in the input area 2201 is displayed on the operation unit 220 using the soft keyboard. The subsequent S1830 is a process for monitoring the presence or absence of user operation. The process checks at predetermined intervals whether the OK button 2202 on the soft keyboard 2200 has been pressed, and if the OK button 2202 is detected to have been pressed, the process proceeds to S1831. Then, in S1831, after the display screen of the operation unit 220 returns from the soft keyboard 2200 to the filename setting screen 1000, the entered string is added to the filename.

[0135] In S1814, it is determined whether the detected user operation is the pressing of the delete button 1003. If the delete button 1003 is pressed, the process proceeds to S1832; otherwise, the process proceeds to S1815. In S1832, the last string of the string entered in the filename input field 1001 is deleted. For example, if the filename structure used to generate the initial filename is ["first character area", "separator", "second character area"], pressing the delete button 1003 deletes the string corresponding to the "second character area". As a result, the filename structure after deletion is ["first character area"], and the filename is only the string corresponding to the "first character area". At this time, the character area corresponding to the deleted string returns to the state indicating that no character area is selected in the preview area 1002. Then, information related to the changed filename structure is stored.

[0136] In S1815, it is determined whether the detected user action is a press of the send button 1007. If the send button 1007 is pressed, the process terminates and proceeds to the next step. Figure 12 S1210 in the flowchart.

[0137] The above describes the filename setting process performed in S1209.

[0138] <Variation Example 1>

[0139] If, following the initial magnification determined based on the leftmost and rightmost character regions in S1802 (determining the initial magnification) above, the average height of each character region displayed in the preview is less than a predetermined threshold, the initial magnification can be changed so that the average height becomes greater than or equal to the predetermined threshold. In this case, the "average height" can be the average of all character regions, or it can be the average of a subset of these character regions. Examples of the average of a subset of these character regions include the average height of the character regions to be displayed in the preview area 1002 at the determined initial magnification, and the average height of the character regions in a specific part of the scanned image (e.g., the upper half), etc. A specific example is given. Here, it is assumed that the interval (width) between the leftmost end of the leftmost character region and the rightmost end of the rightmost character region is 2,000 pixels, the width of the preview area 1002 is 800 pixels, and the predetermined threshold is 15 pixels. Here, the calculated initial multiplier is 0.4 (=800÷2,000). With an average height of 60px for all character regions, the average height of all character regions is 24px (=60×0.4), which is not less than the predetermined threshold. Therefore, 0.4 is used as the initial multiplier. On the other hand, with an average height of 30px for all character regions, the average height of all character regions is 12px (=30×0.4). In this case, the average height of all character regions is less than the predetermined threshold. Therefore, the initial multiplier is changed to 0.5 (=0.4×15÷12), making the average value greater than or equal to the predetermined threshold (here, the average value is the same as the threshold value). Figure 25 This example illustrates the initial display of the filename setting screen 1000 when the average height of the character areas does not meet a threshold and the initial magnification is adjusted. In this example, after adjusting to increase the initial magnification value, the display portion is determined in S1803 based on the left edge of the leftmost character area and the top edge of the topmost character area. As a result, the area corresponding to the right side of the business form is no longer included in the display portion, but the visibility of each character area present in the preview area 1002 is ensured.

[0140] <Variation Example 2>

[0141] In the aforementioned S1803 (determining the initial display portion), if multiple character regions exist for the initial filename, the display portion can be determined such that the topmost character region of the used character regions is displayed in the preview area 1002. Furthermore, the meaning and type of the strings in each character region can be determined, and the display portion can be adjusted so that character regions most likely to be used for the filename are displayed in the preview area 1002. Additionally, the display portion can be determined such that character regions corresponding to the number of pre-specified character regions are displayed in the preview area 1002. Furthermore, the display portion can be determined such that the midpoint between the topmost and bottommost character regions of the character regions used for the initial filename string is located in the center of the preview area 1002.

[0142] <Variation Example 3>

[0143] In the above-described S1805 (display of operation guide), if an initial filename based on a filename structure similar to a business form is automatically entered in the filename input field 1001, the operation guide can also be displayed in a way that does not overlap with the character area of ​​the string used for the initial filename. In this case, consider the possibility that there are multiple strings used for the initial filename, and it is difficult to display the operation guide in a way that does not overlap with these character areas. In such cases, the character area of ​​the string at the beginning of the initial filename can be prioritized, and the operation guide can be displayed in a way that partially or completely overlaps with the character area of ​​the string at the end. Furthermore, if document attributes other than the initial filename (e.g., metadata) are set, the character areas that should avoid overlapping with the operation guide can be determined based on the type of the document attribute, and the display position of the operation guide can be determined accordingly. For example, if the value of a document attribute is set to type date, the operation guide can be displayed in a way that does not overlap with the character area of ​​the date in the object document. In this case, it is determined whether the string corresponding to each character area is a date by judging whether the string extracted from the scanned image corresponds to a specific character array. For example, if numbers are entered into “yy”, “mm” and “dd” in the format yy / mm / dd respectively, it can be determined that the string is a date.

[0144] <Variation Example 4>

[0145] In S1822 (display of the editing soft keyboard) described above, instead of displaying the editing soft keyboard while the touched string is being entered, it can also be displayed while the entire filename is being entered in the input area 2201. In this case, to prevent the user from misidentifying the object to be edited, the string is displayed so that the scope of the object to be edited can be known by highlighting the string (changing the string's color, appending a box to the string, etc.) to distinguish it from other strings, and displaying a cursor at the end of the string. Furthermore, instead of displaying the editing soft keyboard in response to a touch operation on the string entered in the filename input field 1001, the editing soft keyboard can also be displayed by waiting for further additional user operation (e.g., pressing a dedicated button not shown).

[0146] <Variation Example 5>

[0147] Furthermore, in this embodiment, upon detecting a user's touch operation on a character area, processing is performed to add or delete the touched character area string from the filename (S1817, S1818). However, there is a situation where the system response is slow, and therefore, time is required from detecting a touch operation until the addition or deletion of the object string is reflected in the filename. In this case, it is assumed that the user mistakenly believes the system is not responding to their own touch operation and repeatedly touches the same character area several times. In this situation, there may be repeated addition and deletion of the same character area string, and the process may not stop as the user expects. For example, if the user touches to delete the object string multiple times, the deletion object string may be entered again. Therefore, to prevent repeated touch operations on the same character area during the addition / deletion process, the system can be designed such that it does not receive touch operations for new additions or deletions of the same character area until the addition / deletion process is complete. At this time, a message indicating that processing is in progress can also be displayed. Thus, during the input operation when setting a filename, the addition or deletion of strings that the user does not expect can be prevented.

[0148] Other embodiments

[0149] The embodiments of the present invention can also be implemented by providing software (programs) that perform the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads and executes the program.

[0150] According to the technology of the present invention, partial editing of undefined filenames and other elements currently displayed on the UI screen can be easily performed. This improves the user's operational efficiency in filename editing tasks.

[0151] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be interpreted in the broadest sense to include all such modifications, equivalent structures, and functions.

Claims

1. An information processing device, comprising: A first control unit is configured to display a UI screen for setting a filename of a file including a scanned image obtained by scanning a document. The UI screen displays a connecting string as the filename to be set, in which multiple strings identified from the scanned image are connected by delimiters between pairs of the multiple strings. A second control unit is configured to, in response to a user operation on a displayed UI screen relating to the first string of the plurality of strings and displaying a connection string of the plurality of strings connected by delimiters between pairs of the plurality of strings, cause the display to display the first string in an editable manner, the first string being neither connected to the delimiter nor to any other string included in the plurality of strings other than the first string. as well as A setting component is used to set the filename by using a connection string that is edited based on the user's edit of the first string displayed in the editable manner.

2. The information processing device according to claim 1, wherein, After the first string is edited, the connection string used to set the filename is displayed on the UI screen.

3. The information processing device according to claim 1, wherein, In response to the user action, the first string is displayed together with the soft keyboard used to edit the first string.

4. The information processing device according to claim 1, wherein, The information processing device is a printing device.

5. The information processing device according to claim 1, wherein, The first string identified from the scanned image corresponds to a character region detected from the scanned image.

6. The information processing device according to claim 1, wherein, The strings other than the first string correspond to one or more character regions detected from the scanned image.

7. The information processing device according to claim 1, wherein, The information processing device includes a scanner, and the scanned image is obtained by scanning the document using the scanner.

8. The information processing device according to claim 1, wherein, The information processing device is able to communicate with the server and send the file along with its filename to the server and store it on the server.

9. An information processing device, comprising: A first control unit is configured to display a UI screen for setting a filename of a file including a scanned image obtained by scanning a document. The UI screen displays a connecting string as the filename to be set, in which multiple strings identified from the scanned image are connected by delimiters between pairs of the multiple strings. A second control unit is configured to, in response to a user operation on a displayed UI screen relating to the first string of the plurality of strings and displaying a connection string of the plurality of strings connected by delimiters between pairs of the plurality of strings, cause the display to show another UI screen, the other UI screen displaying the first string in an editable manner, the first string being neither connected to the delimiter nor to any other string included in the plurality of strings besides the first string; as well as A receiving component is configured to receive from the user edits of the first string displayed in the editable manner on the other UI screen. The filename is set by using a connection string that is edited by the user based on the editing of the first string displayed in the editable manner.

10. The information processing device according to claim 9, wherein, In response to the user action, the first string is displayed together with the soft keyboard used to edit the first string.

11. The information processing device according to claim 9, wherein, The information processing device is a printing device.

12. The information processing device according to claim 9, wherein, The first string identified from the scanned image corresponds to a character region detected from the scanned image.

13. The information processing device according to claim 9, wherein, The strings other than the first string correspond to one or more character regions detected from the scanned image.

14. The information processing device according to claim 9, wherein, The information processing device includes a scanner, and the scanned image is obtained by scanning the document using the scanner.

15. The information processing device according to claim 9, wherein, The information processing device is able to communicate with the server and send the file along with its filename to the server and store it on the server.

16. A control method for an information processing device, comprising: The display shows a UI screen for setting a filename that includes a scanned image obtained by scanning a document. The UI screen displays a concatenated string as the filename to be set, in which multiple strings identified from the scanned image are connected by delimiters between pairs of the multiple strings. In response to a user action on a displayed UI screen relating to the first string of the plurality of strings and showing the concatenation strings connected by delimiters between pairs of the plurality of strings, the display is made to display the first string in an editable manner, the first string being neither connected to the delimiter nor to any other string included in the plurality of strings other than the first string; as well as The filename is set by using a connection string that is edited based on the user's edit of the first string displayed in the editable manner.

17. A control method for an information processing device, comprising: The display shows a UI screen for setting a filename that includes a scanned image obtained by scanning a document. The UI screen displays a concatenated string as the filename to be set, in which multiple strings identified from the scanned image are connected by delimiters between pairs of the multiple strings. In response to a user action on a displayed UI screen that relates to the first string of the plurality of strings and displays the connected strings linked by delimiters between pairs of the plurality of strings, the display shows another UI screen that displays the first string in an editable manner, the first string being neither connected to the delimiter nor to any other string included in the plurality of strings except the first string; as well as The user receives an edit of the first string displayed in the editable manner on the other UI screen. The filename is set by using a connection string that is edited by the user based on the editing of the first string displayed in the editable manner.

18. A non-transitory computer-readable storage medium storing a program for causing a computer to perform a control method of an information processing apparatus according to claim 16 or 17.

Images