Information Processing Apparatus, Information Processing Method, and Program

Abstract

To make it possible to, even in a situation where revision of an electronic document is performed by additionally writing for a printed matter in which a content of the electronic document is printed, reflect a result of the revision to the electronic document in a more suitable manner.SOLUTION: A job control unit 403 identifies an electronic document serving as a printing source of a printed matter from a first image according to a result reading the printed matter. An image processing unit 404 generates a difference image from a difference between a second image according to a result of rendering the identified electronic document and the first image that is read. The image processing unit 404 identifies an instruction relating to a revision additionally written on the printed matter and a character string to be subjected to the revision, on the basis of the difference image. The image processing unit 404 identifies a position of the character string to be subjected to the revision in the electronic document. In addition, the job control unit 403 edits the character string in the electronic document to be subjected to the revision in accordance with a content of the revision.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, with the development of network technology and virtualization technology, by accessing services on a network called so-called cloud services, it has become possible to use various functions without installing software on an information processing apparatus such as a PC. As an example of such cloud services, cloud storage that enables creation, editing, viewing, and sharing of electronic documents on a network can be mentioned. In addition, among cloud storages, there are some that enable viewing of editing history, restoration of an arbitrary version, and extraction of differences between arbitrary versions.

[0003] In addition, files managed on the cloud can be viewed and edited regardless of the location where the user works or the type of terminal device used for the work, as long as the cloud storage can be accessed via a network. Also, due to such characteristics, files managed on the cloud can be shared in real time, for example, even with other users located in a remote location.

[0004] On the other hand, in recent years, for the confirmation and refinement of the content of electronic documents, it is not uncommon for them to be based on printed materials obtained by printing the content of the electronic documents on a recording medium such as paper. Against this background, in recent years, various printers and MFPs (MultiFunction Peripherals) that can print electronic documents managed in cloud storage by cooperating with the cloud storage have been proposed. For example, Patent Document 1 discloses an example of an image processing apparatus that can print electronic documents managed in cloud storage.

Prior Art Documents

Patent Documents

[0005] Patent Document 1 Japanese Patent Application Laid-Open No. 2017-76244 Summary of the Invention Problems to be Solved by the Invention

[0006] In a situation where proofreading and reviewing are performed manually using a printed matter on which the content of an electronic document is printed, as a method for reflecting the results of the proofreading and reviewing on the electronic document, for example, a method of directly editing the electronic document on cloud storage can be mentioned. However, this method requires re-entering electronically the content once written by hand using a keyboard or the like, which may make the work cumbersome. Also, by using the scanning function provided in a scanner or MFP, it is possible to capture the printed matter on which proofreading and reviewing have been performed as electronic data. However, in this case, since the reading result of the printed matter is captured as image data, for example, even if the image data is used, it has been difficult to automatically reflect the results of the proofreading and reviewing on the printed matter in the text data within the electronic document.

[0007] In view of the above problems, an object of the present invention is to enable the results of the review to be reflected in the electronic document in a more suitable manner even in a situation where the electronic document is reviewed by adding to a printed matter on which the content of the electronic document is printed. Means for Solving the Problems

[0008] The information processing apparatus according to the present invention includes Information indicating a location on cloud storage where a printed electronic document is stored is attached as an image a scanner unit that reads a printed matter and generates a first image, and a correction instruction written on the printed matter Based on the information indicating the location stored at the location indicated by the informationAn operation unit that displays a screen for receiving from the user a first instruction for specifying whether to reflect the correction instruction in the electronic document of the printing source, and a second instruction for specifying, from a list of cloud storages available to the user, the cloud storage that will be the transmission destination of the first image, a receiving means for receiving the first image, the first instruction, and the second instruction from the apparatus, and in the first instruction received by the receiving means, it is specified that the correction instruction is not reflected in the electronic document of the printing source, and in the second instruction, when the cloud storage that will be the transmission destination of the first image is specified, a transmission means for transmitting the first image received by the receiving means to the cloud storage specified in the second instruction, and in the first instruction received by the receiving means, it is specified that the correction instruction is reflected in the electronic document of the printing source, and Based on the information indicating the location on cloud storage where the printed electronic document is stored, which is obtained by analyzing the first image received by the receiving means, from the location on the cloud storage when the electronic document of the printing source can be acquired, a generating means for generating a second image by rendering the electronic document of the printing source, and generating a difference image by extracting the difference between the first image received by the receiving means and the generated second image, and by performing an analysis process and an OCR process on the difference image generated by the generating means, recognizing the correction instruction indicated by the proofreading symbol included in the difference image and the text information of the natural language included in the difference image, and further specifying a character string in the electronic document that is the target of reflection of the correction instruction based on the coordinates of the proofreading symbol, and an execution means for executing a correction process based on the correction instruction on the character string in the electronic document specified by the specifying means.

Effects of the Invention

[0009] According to the present invention, even in a situation where proofreading of an electronic document is performed by adding to a printed matter on which the content of the electronic document is printed, the result of the proofreading can be reflected in the electronic document in a more suitable manner.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0012] <System Configuration> Referring to FIG. 1, an example of the system configuration of the information processing system according to the present embodiment will be described. The information processing system according to the present embodiment includes an MFP 100, a terminal device 113, an MFP server 120, and a cloud storage 121. The MFP 100, the terminal device 113, the MFP server 120, and the cloud storage 121 are interconnected via a predetermined network. As a specific example, in the example shown in FIG. 1, the MFP 100 and the terminal device 113 are connected via a LAN 110. Also, the MFP 120 and the cloud storage 121 are connected to the Internet 112. Moreover, the communication via the LAN 110 and the communication via the Internet 112 are mediated by a gateway 111. Thereby, the MFP 100 and the terminal device 113 connected to the LAN 110 can access the MFP server 120 and the cloud storage 121.

[0013] Note that the configuration shown in FIG. 1 is merely an example, and the network configuration is not limited as long as the MFP 100, the terminal device 113, the MFP server 120, and the cloud storage 121 can be interconnected with each other. Also, the network connecting the various components described above, such as the LAN 110 and the Internet 112, may include a plurality of networks, and the type of some of the plurality of networks may be different from the type of other networks.

[0014] The MFP 100 includes a controller unit 101, an operation unit 102, a printer unit 103, and a scanner unit 104. The operation unit 102 schematically shows an interface for receiving an instruction for the MFP 100 from a user and presenting various information to the user. The printer unit 103 prints an image corresponding to electronic data (e.g., an electronic document, etc.) on a recording medium such as paper (hereinafter also referred to as a "sheet"), and outputs a printed matter with the image printed thereon to a predetermined paper discharge destination with respect to the recording medium. The scanner unit 104 reads a printed matter and generates image data by converting the result of the reading (in other words, the optical reading result of the printed matter) into an image signal. The controller unit 101 is connected to the operation unit 102, the printer unit 103, and the scanner unit 104 respectively, and controls their operations. Thereby, various functions of the MFP 100 (that is, functions as a multifunction device) are realized.

[0015] The terminal device 113 schematically shows a print client that issues a print instruction to the MFP 100 using a print protocol such as the LPR (Line Printer Remote) protocol. Further, the terminal device 113 may access other devices (for example, the MFP server 120, the cloud storage 121, etc.) via a network such as a web browser and provide a user interface for the user to operate the other devices. The terminal device 113 can be realized by, for example, a PC (Personal Computer) or the like.

[0016] The gateway 111 corresponds to a so-called network router that mediates (in other words, relays) communication between a plurality of devices connected to different networks. The MFP server 120 is a server that executes various processes related to improving the functionality of the MFP 100. The cloud storage 121 is a so-called file sharing server that transmits and receives files (data) to and from the MFP 100 using a predetermined protocol such as HTTP (Hyper Text Transfer Protocol).

[0017] <Hardware Configuration> With reference to FIG. 2, an example of the hardware configuration of the MFP 100 will be described, paying particular attention to the configuration of the controller unit 101. The controller unit 101 of the MFP100 includes a CPU (Central Processing Unit) 201, a DRAM (Dynamic Random Access Memory) 202, and an I / O controller 203. The controller unit 101 also includes a network I / F 204, a SATA (Serial Advanced Technology Attachment) 205, a panel I / F 206, a printer I / F 207, and a scanner I / F 208. The controller unit 101 may also include a USB (Universal Serial Bus) 212.

[0018] The CPU 201 and the DRAM 202 are connected via a bus. The DRAM 202 is used as a working area for temporarily holding program data representing arithmetic instructions, data to be processed, etc. in the process of the CPU 201 performing various operations. The CPU 201 and the I / O controller 203 are connected via a bus. The I / O controller 203 controls input / output to various devices according to the instructions of the CPU 201.

[0019] The SATA I / F 205 is connected to the I / O controller 203, and a storage device such as a FlashROM 211 is connected to the SATA I / F 205. The FlashROM 211 stores programs for the CPU 201 to realize the functions of the MFP100 and various data such as document files.

[0020] The network I / F 204 is also connected to the I / O controller 203, and a communication device for realizing communication with a predetermined network such as a wired LAN device 210 is connected to the network I / F 204. The CPU 201 controls communication devices such as the wired LAN device 210 via the network I / F 204 to realize communication with other devices via a network such as the LAN 110.

[0021] In addition, a panel I / F 206 is connected to the I / O controller 203. The CPU 201 controls the operation of the operation unit 102 via the panel I / F 206, thereby receiving instructions from the user via the operation unit 102 and presenting various types of information to the user via the operation unit 102.

[0022] Also, a printer I / F 207 is connected to the I / O controller 203. The CPU 201 controls the operation of the printer unit 103 via the printer I / F 207, thereby realizing a process of printing an image on a recording medium (sheet) such as paper and outputting the printed matter to a predetermined paper discharge destination. Also, a scanner I / F 208 is connected to the I / O controller 203. The CPU 201 controls the operation of the scanner unit 104 via the scanner I / F 208, thereby realizing a process of reading a document (printed matter), converting the result of the reading into an image signal, and generating image data.

[0023] Also, a USB I / F 212 may be connected to the I / O controller 203. The CPU 201 controls the operation of various devices connected to the USB I / F 212 via the USB I / F 212. An example of a device connected to the USB I / F 212 is an IC card reader 213. In this case, the CPU 201 controls the operation of the IC card reader 213 via the USB I / F 212, thereby realizing, for example, a process related to user authentication using an IC card.

[0024] The CPU 201 executes processes related to the realization of various functions of the MFP 100 by controlling the operations of various devices as exemplified above. Here, an example of the control when realizing the copy function will be described below.

[0025] The CPU 201 reads predetermined program data from the Flash ROM 211 via the SATA I / F 205 and expands it in the DRAM 202. The CPU 201 controls the operation of the operation unit 102 via the panel I / F 206 according to the program expanded in the DRAM 202, thereby receiving an instruction related to copying from the user via the operation unit 102. When the CPU 201 receives an instruction related to copying, it controls the operation of the scanner unit 104 via the scanner I / F 208, causing the scanner unit 104 to read a document, and acquires image data (electronic data) corresponding to the result of the reading. The CPU 201 causes the acquired image data to be held in a predetermined storage area (for example, the DRAM 202). The CPU 201 transfers the image data held in the predetermined storage area to the printer unit 103 via the printer I / F 207, and instructs the printer unit 103 to execute processing related to printing based on the image data. In response to this instruction, the printer unit 103 prints an image corresponding to the above image data on a recording medium (sheet) such as paper, and ejects a printed matter corresponding to the result of the printing to a predetermined paper ejection destination.

[0026] <Software Configuration> With reference to FIGS. 3 and 4, an example of the software configuration of the information processing system according to the present embodiment will be described below, particularly focusing on each of the MFP 100 and the MFP server 120.

[0027] (Software Configuration of MFP 100) First, with reference to FIG. 3, an example of the software configuration of the MFP 100 will be described. As components related to the realization of the functions of the MFP100, for example, an operation control unit 301, a data storage unit 302, a job control unit 303, an image processing unit 304, a printing processing unit 305, and a reading processing unit 306 are included. Also, as components related to the realization of the functions of the MFP100, a network control unit 307, a TCP / IP control unit 308, an authentication control unit 309, and an MFP server communication unit 310 are included. The processing of each of the series of components shown in FIG. 3 is realized, for example, by the CPU 201 expanding and executing a program stored in a storage area such as the FlashROM 211 in a work area such as the DRAM 202.

[0028] The operation control unit 301 causes the operation unit 102 to display a screen image for receiving instructions from the user and presenting various information to the user, controls the detection of user operations on the screen image, and the execution of processes associated with components such as buttons displayed on the screen.

[0029] The data storage unit 302 executes processes related to storing various data in a storage area such as the FlashROM 211 and reading various data from the storage area. As a specific example, consider the case where a user instructs to change the device settings related to the operation of the MFP100 via the operation unit 102. In this case, the operation control unit 301 recognizes the content of the instruction from the user received by the operation unit 102, and requests the data storage unit 302 to store the set value of the device settings according to the content of the instruction. The data storage unit 302 receives the request from the operation control unit 301 and stores, as the set value of the device settings, a set value corresponding to the content of the instruction from the user in a predetermined storage area (for example, the FlashROM 211, etc.).

[0030] The job control unit 303 performs various controls related to the execution of jobs. The image processing unit 304 processes image data into data in a desired format (for example, data in a format according to the application) according to an instruction from the job control unit 303. The printing processing unit 305 controls the operation of the printer unit 103 via the printer I / F 207 in accordance with an instruction from the job control unit 303, and executes various processes related to printing an image on a recording medium (sheet) such as paper, outputting a printed matter to a predetermined paper discharge destination, and the like. The reading processing unit 306 controls the operation of the scanner unit 104 via the scanner I / F 208 in accordance with an instruction from the job control unit 303, and executes various processes related to reading a document placed on the reading surface of the scanner unit 104.

[0031] The network control unit 307 controls settings related to a network such as an IP address at a predetermined timing such as at system startup or when a setting change is detected, in accordance with setting values held in a predetermined storage area. Thereby, settings related to the operation of components related to communication such as the TCP / IP control unit 308 are controlled. The TCP / IP control unit 308 executes various processes for transmitting and receiving network packets to and from other devices through the network to which the MFP 100 is connected via the network I / F 204.

[0032] The authentication control unit 309 executes processes related to user authentication based on a predetermined authentication method. For example, the authentication control unit 309 may authenticate a user who uses the MFP 100 by collating information read from an IC card by the IC card reader 213 with data related to the user stored in a predetermined storage area by the data storage unit 302.

[0033] The MFP server communication unit 310 causes the network control unit 307 to control communication with the MFP server 120 in accordance with an instruction from the job control unit 303.

[0034] Here, an example of the operation of each component when the copying function is realized will be described below. For example, when the operation control unit 301 receives an instruction from the user to start copying, it instructs the job control unit 303 to start processing related to the realization of the copy function. The job control unit 303 instructs the reading processing unit 306 to read the original document, and acquires the image data generated based on the image signal output according to the result of the reading. In the following description, for convenience, the image corresponding to the result of reading the original document is also referred to as a "scanned image". The job control unit 303 instructs the image processing unit 304 to convert the acquired image data into data in a format suitable for printing the scanned image. Then, the job control unit 303 instructs the printing processing unit 305 to print based on the image data whose format has been converted by the image processing unit 304. Upon receiving this instruction, the printing processing unit 305 causes the printer unit 103 to execute processing related to printing based on the image data specified by the job control unit 303. Thereby, the copy result of the original document is output as a printed matter.

[0035] (Software Configuration of MFP Server 120) Next, with reference to FIG. 4, an example of the software configuration of the MFP server 120 will be described. As components related to the realization of the functions of the MFP server 120, for example, a Web browser communication unit 401, a data storage unit 402, a job control unit 403, an image processing unit 404, a cloud storage communication unit 405, an MFP communication unit 406, and an authentication control unit 407 are included. A series of software programs executed by the MFP server 120 may be held as distributed resources on the cloud, for example. Furthermore, after the program is read into the non-volatile memory allocated as a distributed resource, the processing of each software may be realized by being executed by the computing resources allocated as distributed resources.

[0036] The Web browser communication unit 401 executes processes related to presenting various information to the user and receiving instructions from the user in response to requests from a Web browser operating on another device (e.g., the terminal device 113). Also, the Web browser communication unit 401 controls the execution of processes associated with components such as buttons on the screen presented to the user via the Web browser.

[0037] The data storage unit 402 executes processes related to storing various data in a storage area such as distributed resources on the cloud and reading various data from the storage area. As a specific example, consider the case where a change in device settings related to the operation of the MFP server 120 is instructed via a Web browser. In this case, the Web browser communication unit 401 recognizes the content of the instruction from the user received by the Web browser and requests the data storage unit 402 to store the setting value of the above device settings according to the content of the instruction. The data storage unit 402, upon receiving the request from the Web browser communication unit 401, stores, as the setting value of the above device settings, a setting value corresponding to the content of the instruction from the user in a predetermined storage area (e.g., distributed resources on the cloud).

[0038] The job control unit 403 performs various controls related to the execution of jobs. The image processing unit 404 processes image data into data in a desired format (e.g., data in a format corresponding to the application) according to an instruction from the job control unit 403. The cloud storage communication unit 405 controls communication with the cloud storage 121 using the user's account information stored in a predetermined storage area by the data storage unit 402.

[0039] The MFP communication unit 406 executes various processes related to communication with the MFP 100. As a specific example, the MFP communication unit 406 may execute processes related to receiving requests transmitted from the MFP 100 and notifying various information to the MFP 100.

[0040] The authentication control unit 407 executes processes related to user authentication based on a predetermined authentication method. For example, the authentication control unit 407 may authenticate a user who uses the MFP server 120 by comparing the authentication information received by the Web browser communication unit 401 with the data related to the user stored in a predetermined storage area by the data storage unit 402.

[0041] <User Interface> With reference to FIGS. 5 to 11, an example of the user interface of the information processing system according to the present embodiment will be described.

[0042] (Menu Screen) First, FIG. 5 will be described. FIG. 5 is a diagram showing an example of a menu screen 501 displayed on the operation unit 102 of the MFP 100. The menu screen 501 is a screen for receiving instructions from the user for executing various functions of the MFP 100. When the IC card reader 213 recognizes the IC card presented by the user, the authentication control unit 309 authenticates the target user based on the information read from the IC card, and when the authentication is successful, the menu screen 501 is displayed.

[0043] The button 502 is a user interface for receiving an instruction from the user for executing the copy function. The button 503 is a user interface for receiving an instruction from the user for executing a printing function (hereinafter, also referred to as "cloud print function") using data (for example, electronic documents) stored in the cloud storage 121. The button 504 is a user interface for receiving an instruction from the user for executing a function of reading a document and storing data corresponding to the result of the reading in the cloud storage 121 (hereinafter, also referred to as "cloud scan function").

[0044] (Cloud Print Screen) Next, FIG. 6 will be described. FIG. 6 is a diagram showing an example of a cloud print screen 601 displayed on the operation unit 102 of the MFP 100. The cloud print screen 601 is a screen for receiving instructions from the user regarding various settings of a printing function (cloud print function) that uses data stored in the cloud storage 121. The cloud print screen 601 is displayed on the operation unit 102, for example, when the button 503 is pressed on the menu screen 501.

[0045] The display area 602 is an area where a list of files that can be used as cloud print targets (in other words, candidates for printing targets) is displayed. The list of files displayed in the display area 602 may be displayed so as to be scrolled, for example, by a scroll bar or the like. Also, when a file presented in the list is selected, the file is set as a cloud print target. Note that the candidates for printing targets displayed in the display area 602 may be limited to files associated with the account information of the user identified by authentication. Also, when a plurality of cloud storages are used as the cloud storage 121, for example, the files stored in each of the plurality of cloud storages may be displayed in the display area 602 as a common list. In this case, information regarding the cloud storage in which the file is stored (for example, cloud storage name, etc.) may be displayed in association with each file.

[0046] The check box 603 is a user interface for receiving an instruction from the user as to whether or not to associate information for specifying a path to the file (hereinafter also referred to as "URL information") with the file selected as a printing target. By associating this URL information with the file, for example, another device such as the MFP 100 can identify the file (the file that is the source of the received file copy) managed by the cloud storage 121 based on the URL information.

[0047] The button 604 is a user interface for receiving an instruction from the user regarding the execution of the cloud printing function according to the instructions received from the user by each of the display area 602 and the checkbox 603. When the button 604 is pressed, information according to the instructions received from the user by each of the display area 602 and the checkbox 603 is transmitted to the MFP server 120, and the MFP server 120 is instructed to execute various processes related to the realization of the cloud printing function.

[0048] The display area 605 is a display area where notification information (for example, a message or the like) for notifying the user of various situations is displayed. As a specific example, when some error occurs and the execution of the process related to printing is interrupted, information regarding the error may be displayed in the display area 605, and the situation at that time may be fed back to the user.

[0049] (Cloud Scan Screen) Next, FIG. 7 will be described. FIG. 7 is a diagram showing an example of a cloud scan screen 701 displayed on the operation unit 102 of the MFP 100. The cloud scan screen 701 is a screen for receiving an instruction from the user regarding various settings of a function (cloud scan function) for reading a document and storing data according to the result of the reading in the cloud storage 121. The cloud scan screen 701 is displayed on the operation unit 102, for example, when the button 504 is pressed on the menu screen 501.

[0050] The display area 702 is an area where a list of cloud storages available as a transmission destination of a file (for example, a file of a scanned image) according to the reading result of the document is displayed. The list of cloud storages displayed in the display area 702 may be displayed so as to be scrollable by, for example, a scroll bar or the like. Also, when the cloud storage presented in the list is selected, the cloud storage is set as the transmission destination of the file according to the reading result of the document. In addition, a list of cloud storages available to the user identified by authentication may be displayed on the display area 702. In this case, for example, the MFP 100 may refer to information related to the connection to the cloud storage for each user registered in the MFP server 120 to identify a list of cloud storages available to the user.

[0051] The check box 703 is a user interface for receiving an instruction from the user as to whether to use the URL information associated with the file selected as the print target when executing the above-described cloud print for the process according to the scan result. By checking the check box 703, for example, based on the above URL information, the cloud storage serving as the transmission destination of the file according to the reading result of the document is specified, and the scan result is reflected in the original file.

[0052] The button 704 is a user interface for receiving an instruction from the user regarding the execution of the cloud scan function according to the instructions from the user received by each of the display area 702 and the check box 703. When the button 704 is pressed, the document is read under the control of the reading processing unit 306, and data (file) according to the result of the reading is transmitted to the MFP server 120. At this time, information according to the instructions from the user received by each of the display area 702 and the check box 703 is transmitted to the MFP server 120 in association with the data according to the reading result of the document.

[0053] The display area 705 is a display area on which notification information (for example, a message or the like) for notifying various situations to the user is displayed. As a specific example, when some error occurs and the execution of the process related to the reading of the document is interrupted, the information regarding the error may be displayed on the display area 705 to feedback the situation at that time to the user.

[0054] (Login screen) Next, FIG. 8 will be described. FIG. 8 is a diagram showing an example of a login screen 801 that the MFP server 120 causes to be displayed on the display unit (e.g., the screen of a browser, etc.) of another device (e.g., the terminal device 113) via a network. The login screen 801 is a screen for receiving from the user an input of information that the MFP server 120 uses for user authentication. The login screen 801 is displayed on the display unit of an information processing device having a communication function, such as the terminal device 113, when the MFP server 120 is accessed using the information processing device in a state where user authentication has not been performed.

[0055] Each of the input areas 802 and 803 is a user interface for receiving from the user an input of information that the MFP server 120 uses for authenticating the user when the user logs in to the MFP server 120. Specifically, in the example shown in FIG. 8, the input area 802 is an input area for receiving from the user an input of identification information (so-called user ID) for identifying each user. Also, the input area 803 is an input area for receiving from the user an input of a password used for authenticating the user corresponding to the identification information input to the input area 802.

[0056] The button 804 is a user interface for receiving from the user an instruction related to logging in to the MFP server 120. When the button 804 is pressed, the information input to each of the input areas 802 and 803 is transmitted to the MFP server 120, and a user authentication based on the information is requested from the MFP server 120. In response to this request, the authentication control unit 407 of the MFP server 120 authenticates the user by comparing the information transmitted together with the request with the user's account information stored in a predetermined storage area by the data storage unit 402. Then, when the authentication is successful, the MFP server 120 permits access to the target user.

[0057] (Menu screen) Next, FIG. 9 will be described. FIG. 9 is a diagram showing an example of a menu screen 901 that the MFP server 120 causes to be displayed on the display unit (e.g., the screen of a browser, etc.) of another device (e.g., the terminal device 113) via a network. The menu screen 901 is a screen for receiving various instructions related to the use of the MFP server 120 from the user. The menu screen 901 is displayed on the display unit of the information processing device (e.g., the terminal device 113) when user authentication is performed based on the information input on the login screen 801 shown in FIG. 8 and the authentication is successful.

[0058] Button 902 is a user interface for receiving an instruction from the user regarding the display of a screen for checking the history of processing in which the MFP server 120 is involved, such as the above-described cloud scan. An example of such a screen will be separately described later with reference to FIG. 10.

[0059] Button 903 is a user interface for receiving an instruction from the user regarding the display of a screen for receiving various settings related to processing in which the MFP server 120 is involved, such as the above-described cloud scan. An example of such a screen will be separately described later with reference to FIG. 11.

[0060] (History Confirmation Screen) Next, FIG. 10 will be described. FIG. 10 is a diagram showing an example of a history confirmation screen 1001 that the MFP server 120 causes to be displayed on the display unit (e.g., the screen of a browser, etc.) of another device (e.g., the terminal device 113) via a network. The history confirmation screen 1001 is a screen for checking the history of processing in which the MFP server 120 is involved, such as the above-described cloud scan. In the present embodiment, the history confirmation screen 1001 is used for checking the history related to the execution of the cloud scan. The history confirmation screen 1001 is displayed on the display unit of the information processing device (e.g., the terminal device 113) when the button 902 is pressed on the menu screen 901 shown in FIG. 9.

[0061] The display area 1002 is a display area where the history related to the execution of the cloud scan is displayed. As the history, for example, information related to the target file, information related to the execution date and time, and information regarding whether the process related to storage in the cloud storage was successful, etc. are displayed for each cloud scan process executed.

[0062] As described above, regarding the information related to the error when an error occurs in the MFP 100, it is displayed in the display area 705 of the cloud scan screen 701 displayed on the operation unit 102 of the MFP 100. On the other hand, regarding the information related to the error when an error occurs in the MFP server 120, it is displayed in the display area 1002 of the history confirmation screen 1001. With such a configuration, it is not necessary to occupy the MFP 100 after the process executed in the MFP 100 is completed, and thereafter, other users can use the MFP 100 for other purposes.

[0063] The button 1003 is a user interface for receiving an instruction from the user related to the call of the screen (for example, the menu screen 901) that is the call source of the history confirmation screen 1001. When the button 1003 is pressed, the screen that is the call source of the history confirmation screen 1001 is displayed again.

[0064] (Settings screen) Next, FIG. 11 will be described. FIG. 11 is a diagram showing an example of a setting screen 1101 that the MFP server 120 causes to be displayed on the display unit (e.g., the screen of a browser, etc.) of another device (e.g., the terminal device 113) via a network. The setting screen 1101 is a screen for receiving from the user various settings related to the processing in which the MFP server 120 is involved, such as the above-described cloud scan. In the present embodiment, it is assumed that the setting screen 1101 is used for specifying settings related to the cloud storage that is used for storing a file according to the reading result of the document when executing the cloud scan. The setting screen 1101 is displayed on the display unit of the information processing device (e.g., the terminal device 113), for example, when the button 903 is pressed on the menu screen 901 shown in FIG. 9.

[0065] The setting area 1102 is a user interface for receiving from the user the specification of information regarding the account used for accessing the cloud storage when executing the cloud scan. In the example shown in FIG. 11, in the setting area 1102, for each cloud storage, an area for receiving the specification of the user ID used for accessing the cloud storage and an area for receiving the specification of the password used for authenticating the corresponding user are provided.

[0066] The button 1003 is a user interface for receiving from the user an instruction regarding the reflection of the information specified in the setting area 1102. When the button 1003 is pressed, the information specified in the setting area 1102 is stored in a predetermined storage area by the data storage unit 402. Note that the information stored in the storage area is used for authenticating the target user when the MFP server 120 accesses the cloud storage 121.

[0067] <Processing> With reference to FIGS. 12 and 13, an example of the processing of the information processing system according to the present embodiment will be described.

[0068] (Cloud Print Function) First, with reference to FIG. 12, an example of the processing related to the realization of the cloud printing function will be described. The series of processes shown in FIG. 12 is started, for example, when the button 604 on the cloud printing screen 601 shown in FIG. 6 is pressed.

[0069] In S1201, the operation control unit 301 identifies the file designated as the printing target by the user according to the selection state of the files displayed as a list in the display area 602 of the cloud printing screen 601. The operation control unit 301 notifies the MFP server communication unit 310 of the file designated as the printing target and the input state of the check box 603, and requests the MFP server communication unit 310 to execute the processing related to the realization of the cloud printing function.

[0070] In S1202, the MFP server communication unit 310 transmits the file identified in S1201 and the information corresponding to the input state of the check box 603 to the MFP server 120. As described above, the MFP server communication unit 310 requests the MFP server 120 to acquire the identified file, and instructs whether to associate URL information with the file according to the input state of the check box 603. Then, the MFP server communication unit 310 determines whether the transmission of the above request to the MFP server 120 was successful. If the MFP server communication unit 310 determines in S1202 that the transmission of the above request to the MFP server 120 has failed, the process proceeds to S1209. In S1209, the operation control unit 301 of the MFP 100 displays information regarding the error that occurred in the display area 605 of the cloud printing screen 601. Thereafter, the series of processes shown in FIG. 12 ends. On the contrary, if the MFP server communication unit 310 determines in S1202 that the transmission of the above request to the MFP server 120 was successful, the process proceeds to S1203.

[0071] In S1203, the MFP communication unit 406 receives the request transmitted from the MFP 100 in S1202, and registers the processing content corresponding to the request with the job control unit 403. The job control unit 403 requests the cloud storage communication unit 405 to perform processing according to the registered processing content. Upon receiving this request, the cloud storage communication unit 405 attempts to acquire the file specified as the acquisition target in S1202 from the cloud storage 121. At this time, the cloud storage communication unit 405 uses the account information corresponding to the target cloud storage stored in a predetermined storage area by the data storage unit 402 for authentication to access the cloud storage. Then, the cloud storage communication unit 405 determines whether the acquisition of the target file was successful. If the cloud storage communication unit 405 determines in S1203 that the acquisition of the target file has failed, the process proceeds to S1209. In S1209, the cloud storage communication unit 405 notifies the MFP 100 that the acquisition of the target file has failed. Upon receiving this notification, the operation control unit 301 of the MFP 100 causes information regarding the error that occurred in the display area 605 of the cloud print screen 601 to be displayed. Thereafter, the series of processes shown in FIG. 12 ends. On the other hand, if the cloud storage communication unit 405 determines in S1203 that the acquisition of the target file was successful, the process proceeds to S1204.

[0072] In S1204, the job control unit 403 determines whether to associate URL information with the target file according to the processing content registered in S1203.

[0073] When the job control unit 403 determines in S1204 that URL information is to be associated with the target file, the process proceeds to S1205. In S1205, the job control unit 403 requests the image processing unit 404 to create a rendering image with the URL information attached. The image processing unit 404 creates a rendering image by performing rendering on the file to be printed obtained in S1203. Examples of the file to be printed include electronic documents in text format and electronic documents described based on a predetermined markup language.

[0074] Examples of the markup language used for describing electronic documents include XML (Extensible Markup Language) and HTML (Hyper Text Markup Language). Of course, these are just examples, and other markup languages may be used as long as they can be used for describing electronic documents. Examples of the file format of the rendering image include bitmap, but a file format with an arbitrary print layout such as PDF (Portable Document Format) may also be applied.

[0075] The image processing unit 404 adds a cover to the created rendering image and attaches, in a predetermined description format, URL information indicating the location where the file to be printed is stored on the cloud storage to the cover. Examples of the description format include a character string indicating the URL or directory path to the target file and a so-called barcode format such as a 2D barcode.

[0076] On the other hand, if the job control unit 403 determines in S1204 not to associate URL information with the target file, the process proceeds to S1208. In S1208, the job control unit 403 requests the image processing unit 404 to create a rendering image with URL information attached. The image processing unit 404 creates a rendering image by performing rendering on the file to be printed obtained in S1203. Note that the process related to the creation of the rendering image is the same as the process in S1205.

[0077] In S1206, the job control unit 403 transmits the rendering image created by the image processing unit 404 in S1205 or S1208 to the MFP 100 via the MFP communication unit 406.

[0078] In S1207, the MFP server communication unit 310 of the MFP 100 receives the rendering image transmitted by the MFP server 120 in S1206, and registers a print job with the rendering image as the print target with the job control unit 303. The job control unit 303 causes the print processing unit 305 to execute the process related to the printing of the rendering image based on the registered print job.

[0079] (Cloud Scan Function) Next, with reference to FIG. 13, an example of the process related to the realization of the cloud scan function will be described. The series of processes shown in FIG. 13 is started, for example, when the button 704 on the cloud scan screen 701 shown in FIG. 7 is pressed.

[0080] In S1301, the operation control unit 301 registers the processing content of the cloud scan with the job control unit 303 according to the selection state of the cloud storage in the display area 702 of the cloud scan screen 701 and the input state with respect to the check box 703. Note that the selection state of the cloud storage in the display area 702 and the input state of the check box 703 may be registered, for example, as attributes of the above processing content. The job control unit 303 requests the reading process unit 306 to execute a process related to reading the document placed on the scanner unit 104 according to the registered above processing content. In response to this request, the reading process unit 306 controls the operation of the scanner unit 104 to execute a process related to reading the document placed on the reading surface of the scanner unit 104. The scanned image obtained as the result of reading the document is output, for example, as data (file) in a bitmap format in which information obtained by optically reading the document is arranged two-dimensionally.

[0081] In S1302, the MFP server communication unit 310 transmits data corresponding to the result of reading the document in S1301 (that is, data of the scanned image) to the MFP server 120 as a processing request. At this time, information regarding the cloud storage selected in the display area 702 and the input state of the check box 703 are also notified to the MFP server 120. Then, the MFP server communication unit 310 determines whether the transmission of the data corresponding to the result of reading the document to the MFP server 120 has been successful. If the MFP server communication unit 310 determines in S1302 that the transmission of the data corresponding to the result of reading the document to the MFP server 120 has failed, the process proceeds to S1319. In S1319, the operation control unit 301 of the MFP 100 causes information regarding the error that has occurred in the display area 705 of the cloud scan screen 701 to be displayed. Then, the series of processes shown in FIG. 13 ends. On the other hand, if the MFP server communication unit 310 determines in S1302 that the transmission of the data corresponding to the result of reading the document to the MFP server 120 has been successful, the process proceeds to S1303.

[0082] In S1303, the MFP communication unit 406 receives the processing request transmitted from the MFP 100, and registers the processing content corresponding to the processing request with the job control unit 403. The job control unit 403 determines whether the destination of the data corresponding to the reading result of the document is specified as a cloud storage based on the information registered as an attribute in the processing request, or whether the association of URL information with the data is specified.

[0083] In S1303, when the job control unit 403 determines that the association of URL information with the data corresponding to the reading result of the document is not specified and the destination cloud storage is specified, the process proceeds to S1321. In S1321, the job control unit 403 transmits the data corresponding to the reading result of the document (in other words, the data of the scanned image) to the cloud storage specified as the above destination via the cloud storage communication unit 405. Note that in this case, since the association of URL information with the above data is not performed, the file on the cloud storage to be reflected by the data is not specified. Therefore, the above data may be placed in, for example, the directory set as the default in the destination cloud storage. Also, the file name of the above data may be determined based on, for example, the conditions set in the cloud storage. As a specific example, the file name of the above target data may be determined based on the information regarding the date and time when the scan was performed. Also, the file format of the above data may be determined based on, for example, the conditions set in the cloud storage. As a specific example, the above target data may be stored as a PDF format file. Of course, the above is only an example and may be changed as appropriate.

[0084] On the other hand, when the job control unit 403 determines in S1303 that the association of URL information with the data according to the reading result of the document is specified, the process proceeds to S1304. In this embodiment, for the sake of convenience, it is assumed that the operation control unit 301 performs exclusive control over the selection state of the cloud storage in the display area 702 and the input state of the checkbox 703 on the cloud scan screen 701. Specifically, when a check is input to the checkbox 703, it is controlled so as to be recognized as a state where no cloud storage is selected for the display area 702. In other words, the input state of the checkbox 703 is given priority over the selection state of the cloud storage in the display area 702.

[0085] In S1304, the job control unit 403 causes the image processing unit 404 to analyze the above-mentioned target data, and based on the result of the analysis, attempts to specify a URL for accessing a file (electronic document) on the cloud storage corresponding to the data. For example, the image processing unit 404 may recognize URL information given in a predetermined description format (for example, the format of a two-dimensional barcode, etc.) for the cover added to the above-mentioned target data in S1205, and specify the above-mentioned URL based on the URL information. As a specific example, image analysis may be performed on a partial image corresponding to a predetermined area of the document (printed matter) among the scanned images according to the reading result of the document, and the URL information presented in the partial image may be recognized. Note that the scanned image according to the reading result of the above-mentioned document corresponds to an example of the "first image". Also, the process of S1304 corresponds to an example of the "first specifying process".

[0086] When the above URL is specified, the job control unit 403 accesses the cloud storage corresponding to the URL via the cloud storage communication unit 405 and attempts to obtain the file (electronic document) corresponding to the above data from the cloud storage. At this time, the job control unit 403 may use, for example, the account information corresponding to the target cloud storage stored in a predetermined storage area by the data storage unit 402 for authentication to access the cloud storage. Then, the job control unit 403 determines whether the file on the cloud storage corresponding to the target data has been obtained.

[0087] If the job control unit 403 determines in S1304 that the file on the cloud storage corresponding to the target data cannot be obtained, the process proceeds to S1320. As a specific example, when the access to the cloud storage fails, when the authentication fails, or when the corresponding file does not exist on the cloud storage, the acquisition of the target file fails in S1304. In this case, the job control unit 403 becomes the target of the process related to reflecting the proofreading result shown in the manuscript based on the data according to the reading result of the manuscript, the details of which will be described separately as the processes of S1305 to S1317, and the file cannot be obtained.

[0088] In S1320, the job control unit 403 instructs the data storage unit 402 to store the history related to the job process in a predetermined storage area. In this case, information indicating the failure of the process related to the cloud scan is stored as the history related to the job process. Then, the series of processes shown in FIG. 13 ends.

[0089] On the other hand, when the job control unit 403 determines in S1304 that the file on the cloud storage corresponding to the data targeted in S1304 has been acquired, the process proceeds to S1305. In S1305, the job control unit 403 requests the image processing unit 404 to create a rendering image of the file (telecommunication document) acquired in S1304. In response to this request, the image processing unit 404 generates a rendering image by performing rendering on the file. In this embodiment, for the sake of convenience, it is assumed that the file is a text-formatted telecommunication document or a telecommunication document described based on a predetermined markup language. Also, it is assumed that the data of the rendering image is in bitmap format. Note that the above-mentioned rendering image corresponds to an example of the "second image".

[0090] In S1306, the image processing unit 404 compares the scanned image according to the result of the manuscript reading in S1301 with the rendering image created in S1305, and acquires a difference image according to the result of the comparison. Note that the scanned image and the rendering image may have different sizes and resolutions. In such a case, for example, the image processing unit 404 may adjust the size and resolution of at least one of the scanned image and the rendering image. As a specific example, when creating the rendering image, the image processing unit 404 may control the size and resolution of the created rendering image according to the size and resolution of the scanned image. Also, the image processing unit 404 may adjust the inclination of at least one of the scanned image and the rendering image assuming a situation where the rendering image has an inclination with respect to the scanned image. As a specific example, the image processing unit 404 may extract the difference from the scanned image while rotating the rendering image, and specify the rotation angle at which the difference becomes minimum as the rotation angle of the rendering image at the time of comparison between the scanned image and the rendering image.

[0091] In S1307, the image processing unit 404 recognizes the difference image obtained in S1306 as an image indicating writing performed on a printed matter of an electronic document that is the source of the rendering image (hereinafter also referred to as a "writing image"). The image processing unit 404 decomposes the writing image into proofreading marks and natural language written as a result of proofreading the printed matter of the electronic document. Examples of proofreading marks include symbols defined as ISO5776 by the International Organization for Standardization, symbols defined as JIS Z 8208 by Japanese Industrial Standards, and the like. Note that the method for decomposing the writing image into proofreading marks and natural language is not particularly limited. As a specific example, the writing image may be decomposed into proofreading marks and natural language by pattern matching using a recognizer constructed by learning the features of each of the proofreading marks and natural language based on so-called machine learning. In addition, the image processing unit 404 performs various analysis processes (for example, image analysis, etc.) on the proofreading marks and natural language extracted from the writing image by the process related to the decomposition, thereby recognizing the correction instruction indicated by the proofreading marks and text-informing the natural language. The text-information of the natural language can be realized, for example, by OCR (Optical Character Recognition) processing. Note that the natural language written on the printed matter during proofreading is used, for example, in a set with proofreading marks and plays a supplementary role such as presenting correction candidates. In addition, the processes of S1306 and S1307 correspond to an example of the "second specific process".

[0092] In S1308, the image processing unit 404 identifies the coordinates on the rendering image of the proofreading symbol. In the present disclosure, the coordinates on the rendering image refer to the area surrounded by four points represented by the X-axis and the Y-axis on a bitmap evaluated as a two-dimensional array. Since the above-mentioned written image (difference image) is obtained by comparing the scanned image and the rendering image in S1306, it goes without saying that it is possible to identify the correspondence relationship of the positions between the rendering image and the written image.

[0093] In S1309, the image processing unit 404 extracts the area corresponding to the coordinates identified in S1308 from the rendering image. As described above, the rendering image is an image generated by rendering a target electronic document (in other words, an electronic document corresponding to a printed matter written by proofreading). Therefore, it is expected that the string that the user targeted for correction in proofreading is presented in the extracted area.

[0094] In S1310, the image processing unit 404 identifies the string presented in the area on the rendering image extracted in S1309. Note that the data of the area extracted in S1309 is the data of an image in a format such as a bitmap. Therefore, the image processing unit 404 may identify the string presented in the area by performing OCR processing on the above area (that is, an image in a format such as a bitmap), for example. The image processing unit 404 may hold the string identified from the above area in text information form.

[0095] In S1311, the image processing unit 404 searches for the string identified in S1310 (that is, the string presented in the area on the rendering image) from the electronic document of the target of reflection. As a specific example, the image processing unit 404 may search for the above string text-informationized in S1310 from the electronic document based on a string comparison with the text information indicating the content of the electronic document of the target of reflection.

[0096] In S1312, the image processing unit 404 identifies at least some of the candidates of the character strings in the electronic document retrieved in S1311 as reflection candidates. Then, the image processing unit 404 performs processing for controlling the display mode so that the character strings serving as the reflection candidates are displayed in a manner different from other character strings for the electronic document. Note that in the present embodiment, it is assumed that the image processing unit 404 performs processing for highlighting the character strings serving as the above reflection candidates for the target electronic document.

[0097] The processing related to the processing for highlighting the character strings serving as the above reflection candidates may be appropriately changed according to the format of the target electronic document. As a specific example, when an electronic document described based on a predetermined markup language is the target, a process of specifying a highlight attribute for the character strings serving as the above reflection candidates may be performed on the electronic document based on the format of the markup language. Also, as another example, when a text-formatted electronic document is the target, it is preferable to identify the appearance positions of the character strings serving as the above reflection candidates in the electronic document and hold information regarding the appearance positions in a predetermined storage area. Note that examples of the information indicating the appearance positions on the text file include information indicating from which character position to which character position from the beginning of the file. Note that this information indicating the appearance positions is used to specify the positions for performing highlight processing during the rendering of the electronic document.

[0098] As described above, by performing processing related to the control of the display mode such as highlight processing, for example, when the same character strings appear multiple times in an electronic document that is the target of reflection, it becomes possible to identify the candidates of the character strings to be corrected. Note that the details of this process will be described separately later.

[0099] In S1313, the image processing unit 404 generates a rendering image by performing rendering on the electronic document on which the processing for highlighting the character strings serving as the reflection candidates is performed in S1312. For example, when an electronic document described based on a predetermined markup language is the target, rendering processing is performed in the same manner as in the normal case, and highlighting is applied to the area corresponding to the candidate string to be reflected. As another example, when a text-formatted electronic document is the target, during rendering, a process for applying highlighting is executed. Specifically, when generating a rendering image, a process of changing the color of the background of the area corresponding to the string based on information on the appearance position of the candidate string to be reflected is performed. Note that the rendering image generated by performing rendering on the electronic document on which the process related to the control of the display mode such as highlighting is performed, as exemplified above, corresponds to an example of the "third image".

[0100] In S1314, the image processing unit 404 compares the rendering image generated in S1305 with the rendering image generated in S1313. Then, based on the result of the comparison, the image processing unit 404 specifies the coordinates corresponding to the area highlighted in the rendering image generated in S1313 on the rendering image generated in S1305.

[0101] In S1315, the image processing unit 404 determines whether the coordinates of the highlighted area specified in S1314 match the coordinates of the proofreading symbol specified in S1308. Note that the size of the highlighted area and the size of the proofreading symbol area do not necessarily match. Therefore, for example, when the size of the partial area overlapping with the other area in at least one of these areas is equal to or greater than a predetermined ratio (for example, more than half), the image processing unit 404 may determine that the coordinates match between these areas.

[0102] When the image processing unit 404 determines in S1315 that the coordinates of the area subjected to the highlighting process do not match the coordinates of the proofreading symbol, the process proceeds to S1312. In this case, the image processing unit 404 re-identifies other candidates and executes the processes after S1312 for the other candidates.

[0103] On the other hand, when the image processing unit 404 determines in S1315 that the coordinates of the area subjected to the highlighting process match the coordinates of the proofreading symbol, the process proceeds to S1316. Note that the process related to identifying the reflection candidates shown as S1308 to S1315 corresponds to an example of the "third identification process". In S1316, the image processing unit 404 corrects the character string in the electronic document identified as the reflection candidate in S1312 according to the content indicated by the proofreading symbol. As a specific example, if the proofreading symbol is a strikethrough, the image processing unit 404 deletes the character string of the reflection candidate. As another example, if the proofreading symbol means correction to another character string, the image processing unit 404 replaces the character string of the reflection candidate with the other character string.

[0104] In S1317, the job control unit 403 transmits the electronic document corrected in S1316 to the cloud storage via the cloud storage communication unit 405 and instructs the cloud storage to update the electronic document to be reflected. As a result, the electronic document to be reflected stored in the cloud storage is overwritten by the electronic document transmitted from the job control unit 403 (that is, the electronic document corrected in S1316). Note that when the cloud storage manages the version of the target electronic document, the electronic document corrected in the above overwriting process is treated as a new version of the electronic document to be reflected. By performing such management, for example, control to return the electronic document stored in the cloud storage to the version before the above overwriting process, control to present the corrected part in the new version as a difference, etc. become possible.

[0105] In S1318, the job control unit 403 causes the data storage unit 402 to store the history related to the processing of the above-described series of jobs in a predetermined storage area. The information stored as the history includes, for example, information regarding the file name, processing date and time, success or failure of processing, etc. of the electronic document to be reflected, which is displayed on the history confirmation screen 1001 shown in FIG. 10. In this case, information indicating the success of the processing related to cloud scan is recorded as the history of the job processing.

[0106] By applying the above-described processing, for example, even when the result of proofreading is handwritten and added to a printed matter of an electronic document, the content indicated by the result of the proofreading can be automatically corrected for the electronic document stored in the cloud storage according to the content. As a result, for example, it is possible to reflect the result of proofreading on the target electronic document without the user performing a complicated operation of electronically transcribing the content handwritten on the printed matter of the electronic document again using a keyboard or the like.

[0107] <Modification Example> A modification example of the information processing system according to the present embodiment will be described below. In the above-described embodiment, when executing cloud scan, the reflected candidate character string is repeatedly searched for and specified from among a series of character string candidates retrieved from the electronic document. In this modification example, by performing processing related to the control of the display mode on the electronic document so that each of the series of character string candidates retrieved from the electronic document is displayed in a different manner from each other, it is possible to specify the reflected candidate character string without performing repeated searches. Therefore, in this modification example, an example in which the reflected candidate character string can be specified by controlling so that each of the series of character string candidates retrieved from the electronic document is highlighted in a different color will be described below. In addition, in this modification example, the description will focus on the parts different from the above-described embodiment, and detailed description of the parts substantially the same as the above-described embodiment will be omitted.

[0108] FIG. 14 is a flowchart showing an example of the processing of the information processing system according to this modified example, and shows an example of the processing related to the realization of the cloud scan function. The series of processes shown in FIG. 14 is started, for example, when the button 704 on the cloud scan screen 701 shown in FIG. 7 is pressed.

[0109] In S1401, the operation control unit 301 registers the processing content of the cloud scan with the job control unit 303 according to the selection state of the cloud storage in the display area 702 of the cloud scan screen 701 and the input state for the check box 703. The job control unit 303 requests the reading processing unit 306 to execute the processing related to the reading of the document placed on the scanner unit 104 according to the registered processing content. In response to this request, the reading processing unit 306 controls the operation of the scanner unit 104 to execute the processing related to the reading of the document placed on the reading surface of the scanner unit 104.

[0110] In S1402, the MFP server communication unit 310 transmits the data corresponding to the reading result of the document in S1401 (that is, the data of the scanned image) to the MFP server 120 as a processing request. At this time, the information regarding the cloud storage selected in the display area 702 and the input state of the check box 703 are also notified to the MFP server 120. Then, the MFP server communication unit 310 determines whether the transmission of the data corresponding to the reading result of the document to the MFP server 120 is successful. If the MFP server communication unit 310 determines in S1402 that the transmission of the data corresponding to the reading result of the document to the MFP server 120 has failed, the process proceeds to S1419. In S1419, the operation control unit 301 of the MFP 100 displays the information regarding the error that has occurred in the display area 705 of the cloud scan screen 701. Then, the series of processes shown in FIG. 14 ends. On the other hand, if the MFP server communication unit 310 determines in S1402 that the transmission of data corresponding to the reading result of the above manuscript to the MFP server 120 has been successful, the process proceeds to S1403.

[0111] In S1403, the MFP communication unit 406 receives the processing request transmitted from the MFP 100 and registers the processing content corresponding to the processing request with the job control unit 403. Based on the information registered as an attribute in the processing request, the job control unit 403 determines whether the destination cloud storage for transmitting the data corresponding to the reading result of the manuscript has been specified, or whether the association of URL information with the data has been specified.

[0112] If the job control unit 403 determines in S1403 that the association of URL information with the data corresponding to the reading result of the manuscript has not been specified and the destination cloud storage has been specified, the process proceeds to S1421. In S1421, the job control unit 403 transmits the data corresponding to the reading result of the manuscript (in other words, the data of the scanned image) to the specified cloud storage as the above destination via the cloud storage communication unit 405.

[0113] On the other hand, if the job control unit 403 determines in S1403 that the association of URL information with the data corresponding to the reading result of the manuscript has been specified, the process proceeds to S1404.

[0114] In S1404, the job control unit 403 causes the image processing unit 404 to analyze the target data, and based on the result of the analysis, attempts to identify the URL for accessing the file (electronic document) on the cloud storage corresponding to the data. Then, the job control unit 403 determines whether the file on the cloud storage corresponding to the target data has been acquired.

[0115] If the job control unit 403 determines in S1404 that the file on the cloud storage corresponding to the target data cannot be acquired, the process proceeds to S1420. As a specific example, if access to the cloud storage fails, authentication fails, or the corresponding file does not exist on the cloud storage, the acquisition of the target file in S1404 fails.

[0116] In S1420, the job control unit 403 instructs the data storage unit 402 to store the history related to the job process in a predetermined storage area. In this case, information indicating the failure of the process related to the cloud scan is recorded as the history related to the job process. Then, the series of processes shown in FIG. 14 ends.

[0117] On the other hand, if the job control unit 403 determines in S1404 that the file on the cloud storage corresponding to the target data can be acquired, the process proceeds to S1405. In S1405, the job control unit 403 requests the image processing unit 404 to create a rendering image of the file (telecommunication document) acquired in S1304. In response to this request, the image processing unit 404 generates a rendering image by performing rendering on the file.

[0118] In S1406, the image processing unit 404 compares the scanned image according to the result of the original document reading in S1401 with the rendering image created in S1405, and acquires a difference image according to the result of the comparison.

[0119] In S1407, the image processing unit 404 recognizes the difference image acquired in S1406 as a writing image indicating the writing performed on the printed matter of the electronic document that is the source of the rendering image.

[0120] In S1408, the image processing unit 404 specifies the coordinates on the rendering image of the proofreading symbol.

[0121] In S1409, the image processing unit 404 extracts a region corresponding to the coordinates specified in S1408 from the rendering image.

[0122] In S1410, the image processing unit 404 identifies the character string presented in the region on the rendering image extracted in S1409.

[0123] In S1411, the image processing unit 404 searches for the character string identified in S1410 from the electronic document to be reflected.

[0124] In S1412, the image processing unit 404 performs processing to control the display mode so that each of the series of candidates of the character string in the electronic document searched in S1411 is displayed in a different manner from other character strings for the target electronic document. At this time, the image processing unit 404 controls the display mode of each candidate so that each of the series of searched candidates is displayed in a different manner. Note that, as described above in this modification example, the image processing unit 404 performs processing for highlighting the character string that is the candidate for the target electronic document. Specifically, the image processing unit 404 performs processing for highlighting each of the series of searched candidates in a different color for the target electronic document. Note that the conditions for color-coding the highlighting for each of the series of searched candidates may be set in advance.

[0125] In S1413, the image processing unit 404 generates a rendering image by performing rendering on the electronic document on which the processing for highlighting the character string corresponding to each of the series of candidates searched in S1412 has been performed.

[0126] In S1414, the image processing unit 404 compares the rendering image generated in S1405 with the rendering image generated in S1413. Then, based on the result of the comparison, the image processing unit 404 identifies the coordinates corresponding to the region highlighted in the rendering image generated in S1413 on the rendering image generated in S1405.

[0127] In S1415, the image processing unit 404 identifies the region that matches the coordinates of the proofreading symbol identified in S1408 among the regions corresponding to each of the series of highlighted candidates identified in S1414. Then, based on the color of the highlight applied to the identified region, the image processing unit 404 identifies the corresponding candidate from among the series of candidates searched in S1412 as the object to be reflected.

[0128] In S1416, the image processing unit 404 corrects the character string in the electronic document identified as the reflection candidate in S1415 according to the content indicated by the proofreading symbol.

[0129] In S1417, the job control unit 403 transmits the electronic document corrected in S1416 to the cloud storage via the cloud storage communication unit 405 and instructs the cloud storage to update the electronic document to be reflected. As a result, the electronic document to be reflected stored in the cloud storage is overwritten by the electronic document transmitted from the job control unit 403 (i.e., the electronic document corrected in S1416).

[0130] In S1418, the job control unit 403 causes the data storage unit 402 to store the history related to the processing of the above series of jobs in a predetermined storage area. The information stored as the history includes, for example, information regarding the file name, processing date and time, success or failure of processing, etc. of the electronic document to be reflected, which is displayed on the history confirmation screen 1001 shown in FIG. 10. In this case, information indicating the success of the processing related to the cloud scan is recorded as the history related to the processing of the job.

[0131] By applying the above-described processing, for example, even when the result of proofreading is handwritten and added to a printed copy of an electronic document, it is possible to automatically perform a correction corresponding to the content indicated by the result of the proofreading on the electronic document stored in the cloud storage. As a result, for example, a user can reflect the result of proofreading on the target electronic document without performing a complicated operation of re-entering electronically, using a keyboard or the like, the content handwritten on the printed copy of the electronic document.

[0132] <Other Embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a recording medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Description of Reference Numerals

[0133] 100 MFP 120 MFP Server 121 Cloud Storage 403 Job Control Unit 404 Image Processing Unit

Claims

Claims 1. A scanner unit that reads a printed matter to which information indicating a location on a cloud storage where an electronic document of a printing source is stored is attached as an image and generates a first image; a first instruction for designating whether to reflect a correction instruction written in the printed matter in the electronic document of the printing source stored at the location indicated by the information; and an operation unit that displays a screen for receiving from the user a second instruction for designating a cloud storage that is a transmission destination of the first image from a list of cloud storages available to the user. A receiving means for receiving the first image, the first instruction, and the second instruction from the apparatus; When it is specified in the first instruction received by the receiving means that the correction instruction is not to be reflected in the electronic document of the printing source, and a cloud storage that is a transmission destination of the first image is specified in the second instruction, a transmitting means for transmitting the first image received by the receiving means to the cloud storage specified in the second instruction; When it is specified in the first instruction received by the receiving means that the correction instruction is to be reflected in the electronic document of the printing source, and the electronic document of the printing source can be acquired from the location on the cloud storage based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, a generating means for generating a second image by rendering the electronic document of the printing source, and generating a difference image by extracting a difference between the first image received by the receiving means and the generated second image; A specifying means for recognizing the correction instruction indicated by a proofreading symbol included in the difference image and text information in natural language included in the difference image by performing an analysis process and an OCR process on the difference image generated by the generating means, and further specifying a character string in the electronic document that is a reflection target of the correction instruction based on the coordinates of the proofreading symbol; An execution means for executing a correction process based on the correction instruction on the character string in the electronic document specified by the specifying means; An information processing apparatus characterized by comprising the above. Claims 2. In the first instruction received by the receiving means, when it is specified to reflect the correction instruction in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source, which is obtained by analyzing the first image received by the receiving means, is stored, when the electronic document of the printing source cannot be acquired from the location on the cloud storage, a storage means for storing information indicating that reflecting the correction instruction in the electronic document of the printing source has failed. The information processing apparatus according to claim 1, further comprising:

3. The correction process executed by the execution means is a process of deleting a character string in the electronic document specified by the specifying means when the correction instruction is deletion. The information processing apparatus according to claim 1, characterized in that:

4. The correction process executed by the execution means is a process of correcting the character string in the electronic document specified by the specifying means to the other character string when the correction instruction is correction to another character string. The information processing apparatus according to claim 1, characterized in that:

5. The electronic document is an electronic document described based on XML or HTML. The information processing apparatus according to claim 1, characterized in that:

6. The location where the electronic document of the printing source of the printed matter is stored is specified by a URL or a directory path described in a character string format or a barcode format in the first image. The information processing apparatus according to claim 1, characterized in that:

7. The electronic document is an electronic document in a text format. The information processing apparatus according to claim 1, characterized in that:

8. An information processing method executed by an information processing apparatus, comprising: A scanner unit that reads a printed matter to which information indicating a location on a cloud storage where an electronic document of a printing source is stored is attached as an image to generate a first image; a first instruction for specifying whether to reflect a correction instruction written in the printed matter in the electronic document of the printing source stored at the location indicated by the information; and an operation unit that displays a screen for receiving from the user a second instruction for specifying a cloud storage to which the first image is to be transmitted from a list of cloud storages available to the user. A receiving step of receiving the first image, the first instruction, and the second instruction from a device comprising: When it is specified in the first instruction received in the receiving step that the correction instruction is not reflected in the electronic document of the printing source, and the cloud storage as the transmission destination of the first image is specified in the second instruction, a transmission step of transmitting the first image received in the receiving step to the cloud storage specified in the second instruction; When it is specified in the first instruction received in the receiving step that the correction instruction is reflected in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received in the receiving step, when the electronic document of the printing source can be acquired from the location on the cloud storage, a second image is generated by rendering the electronic document of the printing source, and a generation step of generating a difference image by extracting the difference between the first image received in the receiving step and the generated second image; A specifying step of recognizing the correction instruction indicated by the proofreading symbol included in the difference image and the text information in natural language included in the difference image by performing an analysis process and an OCR process on the difference image generated in the generating step, and further specifying the character string in the electronic document to be the object of reflecting the correction instruction based on the coordinates of the proofreading symbol; An execution step of executing a correction process based on the correction instruction on the character string in the electronic document specified by the specifying step; An information processing method characterized by comprising the above.

9. A program for causing a computer to function as each means of the information processing apparatus according to any one of Claims 1 to 7.

Images