Information processing apparatus, information processing method, and program

Abstract

To make it possible to, even in a situation where revision of an electronic document is performed by additionally writing for a printed matter in which a content of the electronic document is printed, reflect a result of the revision to the electronic document in a more suitable manner.SOLUTION: A job control unit 403 identifies an electronic document serving as a printing source of a printed matter from a first image according to a result reading the printed matter. An image processing unit 404 identifies an instruction relating to a revision additionally written on the printed matter and a character string to be subjected to the revision, on the basis of a difference image between a second image in accordance with a result of rendering the identified electronic document and a first image that is read. In addition, the image processing unit 404 adds a revised content according to the instruction relating to the revision to the character string to be subjected to the revision in the electronic document, as revision information in software having a function relating to editing of the electronic document.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, with the development of network technology and virtualization technology, by accessing services on a network called so-called cloud services, it has become possible to use various functions without installing software on an information processing apparatus such as a PC. As an example of such cloud services, cloud storage that enables creation, editing, viewing, and sharing of electronic documents on a network can be mentioned. Also, among cloud storages, there are those that enable viewing of editing histories, restoration of arbitrary versions, and extraction of differences between arbitrary versions.

[0003] Also, files managed on the cloud can be viewed and edited regardless of the location where the user works or the type of terminal device used for the work, as long as they can be accessed via the network to the cloud storage. Also, due to such characteristics, files managed on the cloud can be shared in real time even between other users located in remote locations, for example.

[0004] On the other hand, even in recent years, when it comes to checking and refining the content of electronic documents, it is not uncommon to do so based on printed materials obtained by printing the content of the electronic document on a recording medium such as paper. Against this background, in recent years, various printers and MFPs (MultiFunction Peripherals) that can print electronic documents managed in such cloud storage by collaborating with the cloud storage have been proposed. For example, Patent Document 1 discloses an example of an image processing apparatus that can print an electronic document managed in cloud storage.

[0005] In recent years, it has also become possible to recognize characters in an image through image processing such as OCR (Optical Character Recognition) and convert them into data in text format. Against this backdrop, there is also a study on a technology that performs image processing on an image obtained by reading a printed material on which the content of an electronic document is printed using a scanner or the like, recognizes the proofreading result reflected in the printed material, and reflects it in the electronic document.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] On the other hand, characters added by handwriting to a printed material tend to have different shapes depending on the writer. Due to such characteristics, when recognizing handwritten characters through image processing such as OCR, the accuracy of recognition is lower than when recognizing typefaces, and as a result, there may be a situation where the target characters are misrecognized. Therefore, for example, even if a misrecognition occurs in a situation where the proofreading result recognized by image processing on an image corresponding to the reading result of a printed material is reflected in an electronic document, it is conceivable that the result of the misrecognition is directly reflected in the electronic document.

[0008] In view of the above problems, an object of the present invention is to enable the result of proofreading to be reflected in the electronic document in a more suitable manner even in a situation where the proofreading of the electronic document is performed by adding to a printed material on which the content of the electronic document is printed.

Means for Solving the Problems

[0009] The information processing apparatus according to the present invention is Information indicating a location on a cloud storage where an electronic document of a printing source is stored is attached as an imageA scanner unit that reads a printed matter and generates a first image, and the correction instruction written on the printed matter The one stored at the location indicated by the information A first instruction for specifying whether to reflect the correction instruction in the original electronic document of the print source, a second instruction for specifying a cloud storage as the transmission destination of the first image from a list of cloud storages available to the user, and a third instruction for specifying whether to add the correction instruction in a format conforming to the proofreading function of a document editing application capable of editing the original electronic document of the print source. An operation unit that displays a screen for receiving the above from the user, a receiving means for receiving the first image, the first instruction, the second instruction, and the third instruction from the device, and in the first instruction received by the receiving means, it is specified that the correction instruction is not reflected in the original electronic document of the print source, and when the cloud storage as the transmission destination of the first image is specified in the second instruction, a transmission means for transmitting the first image received by the receiving means to the cloud storage specified in the second instruction, and in the first instruction received by the receiving means, it is specified that the correction instruction is reflected in the original electronic document of the print source, and Based on the information indicating a location on a cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, from the location on the cloud storage When the original electronic document of the print source can be obtained, a second image is generated by rendering the original electronic document of the print source, and a difference image is generated by extracting the difference between the first image received by the receiving means and the generated second image. A generating means, by performing an analysis process and an OCR process on the difference image generated by the generating means, recognizing the correction instruction indicated by the proofreading symbol included in the difference image and the text information of the natural language included in the difference image, and further specifying a character string in the electronic document to which the correction instruction is to be reflected based on the coordinates of the proofreading symbol. In the first instruction received by the receiving means, it is specified that the correction instruction is reflected in the original electronic document of the print source, and Based on the information indicating a location on a cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, from the location on the cloud storageWhen it is possible to obtain the electronic document of the printing source and it is specified in the third instruction to add the correction instruction in a format conforming to the proofreading function of the document editing application, the correction content based on the correction instruction is added to the character string in the electronic document specified by the specifying means as a record of the change history in the proofreading function of the document editing application. On the other hand, in the first instruction received by the receiving means, it is specified to reflect the correction instruction in the electronic document of the printing source, and Based on the information indicating a location on a cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, from the location on the cloud storage When it is possible to obtain the electronic document of the printing source and it is specified in the third instruction not to add the correction instruction in a format conforming to the proofreading function of the document editing application, the execution means executes a process of reflecting the correction content based on the correction instruction on the character string in the electronic document specified by the specifying means without recording it as a change history in the proofreading function of the document editing application. It is characterized by comprising:

Effect of the Invention

[0010] According to the present invention, even in a situation where proofreading of an electronic document is performed by adding to a printed matter on which the content of the electronic document is printed, it is possible to additionally reflect the result of the proofreading on the electronic document in a more suitable manner.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0013] <System Configuration> With reference to FIG. 1, an example of the system configuration of the information processing system according to the present embodiment will be described. The information processing system according to the present embodiment includes an MFP 100, a terminal device 113, an MFP server 120, and a cloud storage 121. The MFP 100, the terminal device 113, the MFP server 120, and the cloud storage 121 are interconnected via a predetermined network. As a specific example, in the example shown in FIG. 1, the MFP 100 and the terminal device 113 are connected via a LAN 110. Also, the MFP 120 and the cloud storage 121 are connected to the Internet 112. Moreover, the communication via the LAN 110 and the communication via the Internet 112 are mediated by a gateway 111. Thereby, the MFP 100 and the terminal device 113 connected to the LAN 110 can access the MFP server 120 and the cloud storage 121.

[0014] Note that the configuration shown in FIG. 1 is merely an example, and the network configuration is not limited as long as it is possible to mutually connect the MFP 100, the terminal device 113, the MFP server 120, and the cloud storage 121. Also, the network connecting the various components described above, such as the LAN 110 and the Internet 112, may include a plurality of networks, and the type of some of the plurality of networks may be different from the type of other networks.

[0015] The MFP 100 includes a controller unit 101, an operation unit 102, a printer unit 103, and a scanner unit 104. The operation unit 102 schematically shows an interface for receiving instructions for the MFP 100 from the user and presenting various information to the user. The printer unit 103 prints an image corresponding to electronic data (e.g., an electronic document, etc.) on a recording medium such as paper (hereinafter also referred to as "sheet"), and outputs a printed matter with the image printed thereon to a predetermined paper discharge destination for the recording medium. The scanner unit 104 reads a printed matter and generates image data by converting the result of the reading (in other words, the optical reading result of the printed matter) into an image signal. The controller unit 101 is connected to each of the operation unit 102, the printer unit 103, and the scanner unit 104, and controls their operations. Thereby, various functions of the MFP 100 (that is, functions as a multifunction device) are realized.

[0016] The terminal device 113 schematically shows a print client that issues a print instruction to the MFP 100 using a print protocol such as the LPR (Line Printer Remote) protocol. Also, the terminal device 113 may access other devices (e.g., the MFP server 120, the cloud storage 121, etc.) via a network, such as a Web browser, and provide a user interface for the user to operate the other devices. The terminal device 113 can be realized by, for example, a PC (Personal Computer) or the like.

[0017] The gateway 111 corresponds to a so-called network router that mediates (in other words, relays) communication between a plurality of devices connected to different networks. The MFP server 120 is a server that executes various processes related to improving the functionality of the MFP 100. The cloud storage 121 is a so-called file sharing server that transmits and receives files (data) to and from the MFP 100 using a predetermined protocol such as HTTP (Hyper Text Transfer Protocol).

[0018] <Hardware Configuration> Referring to FIG. 2, an example of the hardware configuration of the MFP 100 will be described with particular attention paid to the configuration of the controller unit 101. The controller unit 101 of the MFP 100 includes a CPU (Central Processing Unit) 201, a DRAM (Dynamic Random Access Memory) 202, and an I / O controller 203. The controller unit 101 also includes a network I / F 204, a SATA (Serial Advanced Technology Attachment) 205, a panel I / F 206, a printer I / F 207, and a scanner I / F 208. The controller unit 101 may also include a USB (Universal Serial Bus) 212.

[0019] The CPU 201 and the DRAM 202 are connected via a bus. The DRAM 202 is used as a working area for temporarily holding program data representing arithmetic instructions, data to be processed, etc. in the process of the CPU 201 performing various operations. The CPU 201 and the I / O controller 203 are connected via a bus. The I / O controller 203 controls input / output to various devices according to instructions from the CPU 201.

[0020] An SATA I / F 205 is connected to the I / O controller 203, and a storage device such as a Flash ROM 211 is connected to the SATA I / F 205. The Flash ROM 211 stores programs for the CPU 201 to implement the functions of the MFP 100 and various data such as document files.

[0021] Also, a network I / F 204 is connected to the I / O controller 203, and a communication device for realizing communication with a predetermined network such as a wired LAN device 210 is connected to the network I / F 204. The CPU 201 controls the communication device such as the wired LAN device 210 via the network I / F 204 to realize communication with other devices via a network such as the LAN 110.

[0022] Also, a panel I / F 206 is connected to the I / O controller 203. The CPU 201 controls the operation of the operation unit 102 via the panel I / F 206 to receive instructions from the user via the operation unit 102 and present various information to the user via the operation unit 102.

[0023] Also, a printer I / F 207 is connected to the I / O controller 203. The CPU 201 controls the operation of the printer unit 103 via the printer I / F 207 to print an image on a recording medium (sheet) such as paper and output the printed matter to a predetermined paper discharge destination. Also, a scanner I / F 208 is connected to the I / O controller 203. The CPU 201 controls the operation of the scanner unit 104 via the scanner I / F 208 to read a document (printed matter), convert the result of the reading into an image signal, and generate image data.

[0024] In addition, a USB I / F 212 may be connected to the I / O controller 203. The CPU 201 controls the operations of various devices connected to the USB I / F 212 via the USB I / F 212. As an example of a device connected to the USB I / F 212, an IC card reader 213 is mentioned. In this case, the CPU 201 controls the operation of the IC card reader 213 via the USB I / F 212 to realize, for example, a process related to user authentication using an IC card.

[0025] The CPU 201 executes processes related to the realization of various functions of the MFP 100 by controlling the operations of various devices as exemplified above. Here, an example of control when realizing the copy function will be described below.

[0026] The CPU 201 reads predetermined program data from the Flash ROM 211 via the SATA I / F 205 and expands it in the DRAM 202. The CPU 201 controls the operation of the operation unit 102 via the panel I / F 206 according to the program expanded in the DRAM 202, and thereby accepts an instruction related to copying from the user via the operation unit 102. When the CPU 201 accepts an instruction related to copying, it controls the operation of the scanner unit 104 via the scanner I / F 208, causes the scanner unit 104 to read a document, and acquires image data (electronic data) corresponding to the result of the reading. The CPU 201 causes the acquired image data to be held in a predetermined storage area (for example, the DRAM 202). The CPU 201 transfers the image data held in the predetermined storage area to the printer unit 103 via the printer I / F 207, and instructs the printer unit 103 to execute a process related to printing based on the image data. In response to this instruction, the printer unit 103 prints an image corresponding to the above image data on a recording medium (sheet) such as paper, and discharges a printed matter corresponding to the result of the printing to a predetermined paper discharge destination.

[0027] <Software Configuration> Referring to FIGS. 3 and 4, an example of the software configuration of the information processing system according to the present embodiment will be described below, particularly focusing on each of the MFP 100 and the MFP server 120.

[0028] (Software Configuration of MFP 100) First, referring to FIG. 3, an example of the software configuration of the MFP 100 will be described. As components related to the realization of the functions of the MFP 100, for example, an operation control unit 301, a data storage unit 302, a job control unit 303, an image processing unit 304, a printing processing unit 305, and a reading processing unit 306 are included. Also, as components related to the realization of the functions of the MFP 100, a network control unit 307, a TCP / IP control unit 308, an authentication control unit 309, and an MFP server communication unit 310 are included. The processing of each of the series of components shown in FIG. 3 is realized, for example, by the CPU 201 expanding and executing a program stored in a storage area such as the Flash ROM 211 in a work area such as the DRAM 202.

[0029] The operation control unit 301 causes the operation unit 102 to display a screen image for receiving instructions from the user and presenting various information to the user, controls the detection of user operations on the screen image, and the execution of processes associated with components such as buttons displayed on the screen.

[0030] The data storage unit 302 executes processes related to storing various data in a storage area such as the Flash ROM 211 and reading various data from the storage area. As a specific example, consider the case where the user instructs to change the device settings related to the operation of the MFP 100 via the operation unit 102. In this case, the operation control unit 301 recognizes the content of the instruction received from the user by the operation unit 102, and requests the data storage unit 302 to store the set value of the device settings according to the content of the instruction. The data storage unit 302 receives the request from the operation control unit 301 and stores, as the set value of the device settings, a set value corresponding to the content of the instruction from the user in a predetermined storage area (for example, the Flash ROM 211, etc.).

[0031] The job control unit 303 performs various controls related to the execution of a job. The image processing unit 304 processes image data into data in a desired format (for example, data in a format according to the application) according to an instruction from the job control unit 303. The printing processing unit 305 controls the operation of the printer unit 103 via the printer I / F 207 according to an instruction from the job control unit 303, and executes various processes related to printing an image on a recording medium (sheet) such as paper, outputting a printed matter to a predetermined paper discharge destination, and the like. The reading processing unit 306 controls the operation of the scanner unit 104 via the scanner I / F 208 according to an instruction from the job control unit 303, and executes various processes related to reading a document placed on the reading surface of the scanner unit 104.

[0032] The network control unit 307 controls settings related to a network such as an IP address at a predetermined timing such as at system startup or when a setting change is detected, according to the set values held in a predetermined storage area. As a result, settings related to the operation of components related to communication such as the TCP / IP control unit 308 are controlled. The TCP / IP control unit 308 executes various processes for transmitting and receiving network packets to and from other devices through the network to which the MFP 100 is connected via the network I / F 204.

[0033] The authentication control unit 309 executes processes related to user authentication based on a predetermined authentication method. For example, the authentication control unit 309 may authenticate a user who uses the MFP 100 by collating the information read from the IC card by the IC card reader 213 with the data related to the user stored in a predetermined storage area by the data storage unit 302.

[0034] The MFP server communication unit 310 causes the network control unit 307 to control communication with the MFP server 120 according to an instruction from the job control unit 303.

[0035] Here, an example of the operation of each component when implementing the copy function will be described below. For example, when the operation control unit 301 receives an instruction related to the start of copying from the user, it instructs the job control unit 303 to start the process related to the implementation of the copy function. The job control unit 303 instructs the reading processing unit 306 to read the original document, and acquires the image data generated based on the image signal output according to the result of the reading. In the following description, for convenience, the image corresponding to the result of reading the original document is also referred to as a "scanned image". The job control unit 303 instructs the image processing unit 304 to convert the acquired image data into data in a format suitable for printing the scanned image. Then, the job control unit 303 instructs the printing processing unit 305 to print based on the image data whose format has been converted by the image processing unit 304. Upon receiving this instruction, the printing processing unit 305 causes the printer unit 103 to execute the process related to printing based on the image data specified by the job control unit 303. As a result, the copy result of the original document is output as a printed matter.

[0036] (Software Configuration of MFP Server 120) Next, with reference to FIG. 4, an example of the software configuration of the MFP server 120 will be described. As components related to the implementation of the functions of the MFP server 120, for example, a Web browser communication unit 401, a data storage unit 402, a job control unit 403, an image processing unit 404, a cloud storage communication unit 405, an MFP communication unit 406, and an authentication control unit 407 are included. A series of software programs executed by the MFP server 120 may be held as distributed resources on the cloud, for example. Moreover, after the program is read into the non-volatile memory allocated as a distributed resource, the processing of each software may be realized by being executed by the computing resources allocated as distributed resources.

[0037] The Web browser communication unit 401 executes processes related to presenting various information to the user and receiving instructions from the user in response to requests from a Web browser operating on another device (e.g., the terminal device 113). Further, the Web browser communication unit 401 controls the execution of processes associated with components such as buttons on the screen presented to the user via the Web browser.

[0038] The data storage unit 402 executes processes related to storing various data in a storage area such as distributed resources on the cloud and reading various data from the storage area. As a specific example, consider the case where an instruction to change device settings related to the operation of the MFP server 120 is given via a Web browser. In this case, the Web browser communication unit 401 recognizes the content of the instruction from the user received by the Web browser and requests the data storage unit 402 to store the setting value of the above device settings according to the content of the instruction. The data storage unit 402, upon receiving the request from the Web browser communication unit 401, stores, as the setting value of the above device settings, a setting value corresponding to the content of the instruction from the user in a predetermined storage area (e.g., distributed resources on the cloud).

[0039] The job control unit 403 performs various controls related to the execution of jobs. The image processing unit 404 processes image data into data in a desired format (e.g., data in a format according to the application) according to an instruction from the job control unit 403. The cloud storage communication unit 405 controls communication with the cloud storage 121 using the user's account information stored in a predetermined storage area by the data storage unit 402.

[0040] The MFP communication unit 406 executes various processes related to communication with the MFP 100. As a specific example, the MFP communication unit 406 may execute processes related to receiving requests transmitted from the MFP 100 and notifying various information to the MFP 100.

[0041] The authentication control unit 407 executes processing related to user authentication based on a predetermined authentication method. For example, the authentication control unit 407 may authenticate a user who uses the MFP server 120 by collating the authentication information received by the Web browser communication unit 401 with the data related to the user stored in a predetermined storage area by the data storage unit 402.

[0042] <User Interface> With reference to FIGS. 5 to 11, an example of the user interface of the information processing system according to the present embodiment will be described.

[0043] (Menu Screen) First, FIG. 5 will be described. FIG. 5 is a diagram showing an example of a menu screen 501 displayed on the operation unit 102 of the MFP 100. The menu screen 501 is a screen for receiving instructions from the user regarding the execution of various functions of the MFP 100. When the IC card reader 213 recognizes the IC card presented by the user, the authentication control unit 309 authenticates the target user based on the information read from the IC card, and when the authentication is successful, the menu screen 501 is displayed.

[0044] The button 502 is a user interface for receiving an instruction from the user regarding the execution of the copy function. The button 503 is a user interface for receiving an instruction from the user regarding the execution of a printing function (hereinafter, also referred to as "cloud print function") using data (for example, electronic documents) stored in the cloud storage 121. The button 504 is a user interface for receiving an instruction from the user regarding the execution of a function (hereinafter, also referred to as "cloud scan function") that reads a document and stores data according to the result of the reading in the cloud storage 121.

[0045] (Cloud Print Screen) Next, FIG. 6 will be described. FIG. 6 is a diagram showing an example of a cloud print screen 601 displayed on the operation unit 102 of the MFP 100. The cloud print screen 601 is a screen for receiving instructions from the user regarding various settings of a printing function (cloud print function) using data stored in the cloud storage 121. The cloud print screen 601 is displayed on the operation unit 102, for example, when the button 503 is pressed on the menu screen 501.

[0046] The display area 602 is an area where a list of files that can be used as cloud print targets (in other words, candidates for printing targets) is displayed. The list of files displayed in the display area 602 may be displayed so as to be scrollable by, for example, a scroll bar or the like. Also, when a file presented in the list is selected, the file is set as a cloud print target. Note that the candidates for printing targets displayed in the display area 602 may be limited to files associated with the account information of the user identified by authentication. Also, when a plurality of cloud storages are used as the cloud storage 121, for example, files stored in each of the plurality of cloud storages may be displayed in the display area 602 as a common list. In this case, information regarding the cloud storage in which the file is stored (for example, cloud storage name, etc.) may be displayed in association with each file.

[0047] The check box 603 is a user interface for receiving an instruction from the user as to whether or not to associate information (hereinafter also referred to as "URL information") for specifying a path to the selected file as a printing target. By associating this URL information with the file, for example, another device such as the MFP 100 can identify the file (the file that is the replication source of the received file) managed by the cloud storage 121 based on the URL information.

[0048] The button 604 is a user interface for receiving an instruction from the user regarding the execution of the cloud printing function according to the instructions received from the user by each of the display area 602 and the checkbox 603. When the button 604 is pressed, information according to the instructions received from the user by each of the display area 602 and the checkbox 603 is transmitted to the MFP server 120, and the MFP server 120 is instructed to execute various processes related to the realization of the cloud printing function.

[0049] The display area 605 is a display area where notification information (for example, a message or the like) for notifying the user of various situations is displayed. As a specific example, when an error occurs and the execution of the process related to printing is interrupted, information regarding the error may be displayed in the display area 605, and the situation at that time may be fed back to the user.

[0050] (Cloud Scan Screen) Next, FIG. 7 will be described. FIG. 7 is a diagram showing an example of a cloud scan screen 701 displayed on the operation unit 102 of the MFP 100. The cloud scan screen 701 is a screen for receiving an instruction from the user regarding various settings of a function (cloud scan function) for reading a document and storing data according to the result of the reading in the cloud storage 121. The cloud scan screen 701 is displayed on the operation unit 102, for example, when the button 504 is pressed on the menu screen 501.

[0051] The display area 702 is an area where a list of cloud storages available as a transmission destination of a file (for example, a file of a scanned image) according to the result of reading a document is displayed. The list of cloud storages displayed in the display area 702 may be displayed so as to be scrolled by, for example, a scroll bar or the like. Further, when the cloud storage presented in the list is selected, the cloud storage is set as the transmission destination of the file according to the result of reading the document. In addition, a list of cloud storages available to the user identified by authentication may be displayed on the display area 702. In this case, for example, the MFP 100 may refer to information related to the connection to the cloud storage for each user registered in the MFP server 120 to identify a list of cloud storages available to the user.

[0052] The check box 703 is a user interface for receiving an instruction from the user as to whether to use the URL information associated with the file selected as the print target when executing the above-described cloud print for the processing according to the scan result. By checking the check box 703, for example, based on the above URL information, the cloud storage that is the transmission destination of the file according to the read result of the document is identified, and the scan result is reflected in the file of the print source.

[0053] The check box 706 is a user interface for receiving an instruction from the user regarding the method of reflection in the case where the information according to the read result of the document is reflected in the file of the print source of the document. For example, by checking the check box 706, the information according to the read result of the document is added as proofreading information without being overwritten on the file of the print source of the document. In the present disclosure, the proofreading information corresponds to information indicating a correction made when operating in the operation mode related to the proofreading of the electronic document in a predetermined software (document editing application) having a function related to the editing of the target electronic document. Also, when the check box 706 is not checked, the information according to the read result of the document will be reflected by overwriting on the file of the print source of the document.

[0054] Button 704 is a user interface for receiving an instruction from the user regarding the execution of the cloud scan function according to the instructions received from the user by each of the display area 702 and the check box 703. When button 704 is pressed, the original document is read under the control of the reading processing unit 306, and data (file) corresponding to the result of the reading is transmitted to the MFP server 120. At this time, information according to the instructions received from the user by each of the display area 702, the check box 703, and the check box 706 is transmitted to the MFP server 120 in association with the data corresponding to the reading result of the original document.

[0055] The display area 705 is a display area where notification information (for example, a message or the like) for notifying the user of various situations is displayed. As a specific example, when an error occurs and the execution of the process related to the reading of the original document is interrupted, information regarding the error may be displayed in the display area 705, and the situation at that time may be fed back to the user.

[0056] <Process> Referring to FIG. 8, an example of the process of the information processing system according to the present embodiment will be described, particularly focusing on the cloud scan function. The series of processes shown in FIG. 8 is started, for example, when the button 704 on the cloud scan screen 701 shown in FIG. 7 is pressed.

[0057] In S801, the operation control unit 301 registers the processing content of the cloud scan in the job control unit 303 according to the selection state of the cloud storage in the display area 702 of the cloud scan screen 701 and the input states of the check boxes 703 and 706. Note that the selection state of the cloud storage in the display area 702 and the input states of the check boxes 703 and 706 may be registered, for example, as attributes of the above processing content. The job control unit 303 requests the reading processing unit 306 to execute processing related to reading of the document placed on the scanner unit 104 according to the registered processing content. In response to this request, the reading processing unit 306 controls the operation of the scanner unit 104 to execute processing related to reading of the document placed on the reading surface of the scanner unit 104. The scanned image obtained as a result of reading the document is output, for example, as data (file) in a bitmap format in which information obtained by optically reading the document is arranged two-dimensionally.

[0058] In S802, the MFP server communication unit 310 transmits data corresponding to the reading result of the document in S801 (that is, data of the scanned image) to the MFP server 120 as a processing request. At this time, information regarding the cloud storage selected in the display area 702 and the input states of the check boxes 703 and 706 are also notified to the MFP server 120. Then, the MFP server communication unit 310 determines whether the transmission of the data corresponding to the reading result of the document to the MFP server 120 has been successful. If the MFP server communication unit 310 determines in S802 that the transmission of the data corresponding to the reading result of the document to the MFP server 120 has failed, the process proceeds to S820. In S820, the operation control unit 301 of the MFP 100 causes information regarding the error that has occurred in the display area 705 of the cloud scan screen 701 to be displayed. Then, the series of processes shown in FIG. 8 ends. On the other hand, if the MFP server communication unit 310 determines in S802 that the transmission of the data corresponding to the reading result of the document to the MFP server 120 has been successful, the process proceeds to S803.

[0059] In S803, the MFP communication unit 406 receives the processing request transmitted from the MFP 100, and registers the processing content corresponding to the processing request with the job control unit 403. The job control unit 403 determines, based on the information registered as an attribute in the processing request, whether the destination of the data corresponding to the reading result of the document is specified as a cloud storage, or whether the association of URL information with the data is specified.

[0060] If, in S803, the job control unit 403 determines that the association of URL information with the data corresponding to the reading result of the document is not specified and the destination cloud storage is specified, the process proceeds to S823. In S823, the job control unit 403 transmits, via the cloud storage communication unit 405, the data corresponding to the reading result of the document (in other words, the data of the scanned image) to the cloud storage specified as the above destination. Note that, in this case, since the association of URL information with the above data is not performed, the file on the cloud storage that is the target of reflection by the data is not specified. Therefore, the above data may be placed, for example, in the directory set as the default in the destination cloud storage. Also, the file name of the above data may be determined, for example, based on the conditions set in the cloud storage. As a specific example, the file name of the above target data may be determined based on the information regarding the date and time when the scan was performed. Also, the file format of the above data may be determined, for example, based on the conditions set in the cloud storage. As a specific example, the above target data may be stored as a PDF format file. Of course, the above is only an example and may be changed as appropriate.

[0061] On the other hand, when the job control unit 403 determines in S803 that the association of URL information with respect to the data according to the reading result of the document is specified, the process proceeds to S804. In the present embodiment, for the sake of convenience, it is assumed that the operation control unit 301 performs exclusive control on the selection state of the cloud storage in the display area 702 and the input state with respect to the check box 703 on the cloud scan screen 701. Specifically, when the check box 703 is checked, it is controlled so as to be recognized that the cloud storage is not selected for the display area 702. In other words, it is assumed that the input state with respect to the check box 703 takes precedence over the selection state of the cloud storage in the display area 702.

[0062] In S804, the job control unit 403 causes the image processing unit 404 to analyze the target data, and based on the result of the analysis, attempts to identify a URL for accessing a file (electronic document) on the cloud storage corresponding to the data. For example, the image processing unit 404 may recognize URL information given in a predetermined description format (for example, the format of a two-dimensional barcode, etc.) for the cover added to the data at the time of cloud printing, and identify the URL based on the URL information. As a specific example, image analysis may be performed on a partial image corresponding to a predetermined area of the document (printed matter) among the scanned images according to the reading result of the document, so as to recognize the URL information presented in the partial image. Note that the scanned image according to the reading result of the document corresponds to an example of the "first image". Also, the process of S804 corresponds to an example of the "first identification process".

[0063] When the above URL is identified, the job control unit 403 accesses the cloud storage corresponding to the URL via the cloud storage communication unit 405, and attempts to obtain the file (electronic document) corresponding to the above data from the cloud storage. At this time, the job control unit 403 may use, for example, the account information corresponding to the target cloud storage stored in a predetermined storage area by the data storage unit 402 for authentication to access the cloud storage. Then, the job control unit 403 determines whether the file on the cloud storage corresponding to the target data has been obtained.

[0064] If the job control unit 403 determines in S804 that the file on the cloud storage corresponding to the target data cannot be obtained, the process proceeds to S821. As a specific example, when the access to the cloud storage fails, the authentication fails, or the corresponding file does not exist on the cloud storage, the acquisition of the target file fails in S804. In this case, the job control unit 403 becomes the target of the process related to reflecting the proofreading result shown in the manuscript based on the data according to the reading result of the manuscript, the details of which will be described separately in the processes of S805 to S818, and the file cannot be obtained.

[0065] In S821, the job control unit 403 instructs the data storage unit 402 to store the history related to the job process in a predetermined storage area. In this case, information indicating the failure of the process related to the cloud scan is stored as the history related to the job process. Then, the series of processes shown in FIG. 8 ends.

[0066] On the other hand, when the job control unit 403 determines in S804 that the file on the cloud storage corresponding to the target data has been acquired, the process proceeds to S805. In S805, the job control unit 403 requests the image processing unit 404 to create a rendering image of the file (telecommunication document) acquired in S804. In response to this request, the image processing unit 404 generates a rendering image by performing rendering on the file. In this embodiment, for the sake of convenience, it is assumed that the file is a text-formatted telecommunication document or a telecommunication document described based on a predetermined markup language. Also, for the sake of convenience, the markup language is XML (Extensible Markup Language), but the available markup language is not necessarily limited, and other markup languages may be used. Also, the data of the rendering image is assumed to be bitmap-formatted data. Note that the above rendering image corresponds to an example of the "second image".

[0067] In S806, the image processing unit 404 compares the scanned image according to the result of the original document reading in S801 with the rendering image created in S805, and acquires a difference image according to the result of the comparison. Note that the scanned image and the rendering image may have different sizes and resolutions. In such a case, for example, the image processing unit 404 may adjust the size and resolution of at least one of the scanned image and the rendering image. As a specific example, when creating the rendering image, the image processing unit 404 may control the size and resolution of the created rendering image according to the size and resolution of the scanned image. Further, assuming a situation where the rendering image has an inclination with respect to the scanned image, the image processing unit 404 may adjust the inclination of at least one of the scanned image and the rendering image. As a specific example, the image processing unit 404 extracts the difference from the scanned image while rotating the rendering image, and may specify the rotation angle at which the difference becomes minimum as the rotation angle of the rendering image when comparing the scanned image and the rendering image.

[0068] In S807, the image processing unit 404 recognizes the difference image acquired in S806 as an image (hereinafter also referred to as "writing image") indicating the writing performed on the printed matter of the electronic document that is the generation source of the rendering image. The image processing unit 404 decomposes the writing image into proofreading marks and natural language written as a result of proofreading the printed matter of the electronic document. Examples of proofreading marks include symbols defined as ISO5776 by the International Organization for Standardization and symbols defined as JIS Z 8208 by the Japanese Industrial Standards. Note that the method of decomposing the writing image into proofreading marks and natural language is not particularly limited. As a specific example, the writing image may be decomposed into proofreading marks and natural language by pattern matching using a recognizer constructed by learning the features of each of the proofreading marks and natural language based on so-called machine learning. Further, the image processing unit 404 performs various analysis processes (for example, image analysis, etc.) on the proofreading marks and natural language extracted from the writing image by the process related to the decomposition, thereby recognizing the correction instruction indicated by the proofreading mark and converting the natural language into text information. The conversion of the natural language into text information can be realized, for example, by OCR processing. Note that the natural language written on the printed matter during proofreading is used, for example, in a set with proofreading marks and plays a supplementary role such as presenting correction candidates. Also, the processes of S806 and S807 correspond to an example of the "second specifying process".

[0069] In S808, the image processing unit 404 identifies the coordinates on the rendering image of the proofreading symbol. In the present disclosure, the coordinates on the rendering image shall refer to the area enclosed by four points represented by the X-axis and Y-axis on a bitmap evaluated as a two-dimensional array. Since the above-written image (difference image) is obtained by comparing the scanned image and the rendering image in S806, it goes without saying that it is possible to identify the correspondence relationship of the positions between the rendering image and the written image.

[0070] In S809, the image processing unit 404 extracts the area corresponding to the coordinates identified in S808 from the rendering image. As described above, the rendering image is an image generated by rendering a target electronic document (in other words, an electronic document corresponding to a printed matter on which writing has been performed by proofreading). Therefore, it is expected that the extracted area presents the character string that the user targeted for correction in proofreading.

[0071] In S810, the image processing unit 404 identifies the character string presented in the area on the rendering image extracted in S809. Note that the data of the area extracted in S809 is the data of an image in a format such as a bitmap. Therefore, the image processing unit 404 may identify the character string presented in the area by performing OCR processing on the above area (that is, an image in a format such as a bitmap), for example. The image processing unit 404 may convert the character string identified from the above area into text information and hold it.

[0072] In S811, the image processing unit 404 searches for the character string identified in S810 (that is, the character string presented in the area on the rendering image) from the electronic document to be reflected. As a specific example, the image processing unit 404 may search for the above character string converted into text information in S810 from the electronic document based on a character string comparison with the text information indicating the content of the electronic document to be reflected.

[0073] In S812, the image processing unit 404 identifies at least some of the candidates of the character strings in the electronic document retrieved in S811 as reflection candidates. Then, the image processing unit 404 performs processing for controlling the display mode so that the character strings serving as the reflection candidates are displayed in a manner different from other character strings for the electronic document. Note that, in the present embodiment, it is assumed that the image processing unit 404 performs processing for highlighting the character strings serving as the above reflection candidates for the target electronic document.

[0074] The processing related to the processing for highlighting the character strings serving as the above reflection candidates may be appropriately changed according to the format of the target electronic document. As a specific example, when an electronic document described based on a predetermined markup language is the target, processing for specifying a highlight attribute for the character strings serving as the above reflection candidates may be performed on the electronic document based on the format of the markup language. Also, as another example, when a text-formatted electronic document is the target, it is advisable to identify the appearance positions of the character strings serving as the above reflection candidates in the electronic document and hold information regarding the appearance positions in a predetermined storage area. Note that, as information indicating the appearance positions on the text file, for example, information indicating from which character position to which character position from the beginning of the file can be cited. Note that this information indicating the appearance positions is used to specify the positions for performing the highlight processing when rendering the electronic document.

[0075] As described above, by performing processing related to the control of the display mode such as the highlight processing, for example, when the same character strings appear multiple times in an electronic document to be reflected, it becomes possible to identify the candidates of the character strings to be corrected. Note that the details of the same processing will be described separately later.

[0076] In S813, the image processing unit 404 generates a rendering image by performing rendering on the electronic document on which the processing for highlighting the character strings serving as the reflection candidates is performed in S812. For example, when an electronic document described based on a predetermined markup language is the target, rendering processing is performed in the same manner as in the normal case, and highlighting is applied to the area corresponding to the string that is a reflection candidate. As another example, when a text-formatted electronic document is the target, during rendering, a process for applying highlighting is executed. Specifically, when generating a rendering image, a process of changing the color of the background of the area corresponding to the string is performed based on information regarding the appearance position of the string that is a reflection candidate. Note that the rendering image generated by performing rendering on the electronic document on which the process related to the control of the display mode such as highlighting is performed, as exemplified above, corresponds to an example of the "third image".

[0077] In S814, the image processing unit 404 compares the rendering image generated in S805 with the rendering image generated in S813. Then, based on the result of the comparison, the image processing unit 404 specifies the coordinates corresponding to the area highlighted in the rendering image generated in S813 on the rendering image generated in S805.

[0078] In S815, the image processing unit 404 determines whether the coordinates of the highlighted area specified in S814 match the coordinates of the proofreading symbol specified in S808. Note that the size of the highlighted area and the size of the proofreading symbol area do not necessarily match. Therefore, for example, the image processing unit 404 may determine that the coordinates match between these areas when the size of the partial area that overlaps with the other area in at least one of these areas is equal to or greater than a predetermined ratio (for example, more than half).

[0079] When the image processing unit 404 determines in S815 that the coordinates of the area subjected to the highlighting process do not match the coordinates of the proofreading symbol, the process proceeds to S812. In this case, the image processing unit 404 identifies other candidates again and executes the processes after S812 for the other candidates.

[0080] On the other hand, when the image processing unit 404 determines in S815 that the coordinates of the area subjected to the highlighting process match the coordinates of the proofreading symbol, the process proceeds to S816. Note that the process related to the identification of the reflection candidates shown as S808 to S815 corresponds to an example of the "third identification process".

[0081] In S816, the job control unit 403 determines whether the specified operation mode is the proofreading mode based on the input state of the check box 706 registered as the processing content of the cloud scan for the job. Specifically, the job control unit 403 determines that it is in the proofreading mode when the check box 706 is checked.

[0082] When the job control unit 403 determines in S816 that the specified operation mode is not the proofreading mode, the process proceeds to S822. In S822, the image processing unit 404 corrects the character string in the electronic document identified as the reflection candidate in S812 according to the content indicated by the proofreading symbol. As a specific example, when the proofreading symbol is a strikethrough, the image processing unit 404 deletes the character string of the reflection candidate. As another example, when the proofreading symbol means correction to another character string, the image processing unit 404 replaces the character string of the reflection candidate with the other character string.

[0083] On the other hand, when the job control unit 403 determines that the operation mode specified in S816 is the proofreading mode, the process proceeds to S817. In S817, the image processing unit 404 adds, as proofreading information, a description corresponding to the correction content indicated by the proofreading symbol to the character string in the electronic document specified as a reflection candidate in S812. For example, in the case of an electronic document in XML format, the image processing unit 404 may add and reflect information indicating the content of the above correction to the electronic document using an XML tag indicating that it is proofreading information. As a more specific example, if the proofreading symbol is a strikethrough, the image processing unit 404 adds it as proofreading information by attaching an XML tag indicating deletion to the character string of the reflection candidate. As another example, if the proofreading symbol means correction to another character string, the image processing unit 404 adds it as proofreading information by attaching an XML tag for correcting the character string of the reflection candidate to the other character string. Also, when the electronic document to be reflected is in the file format of a specific document editing application, the reflection of the proofreading information (correction content) may be performed in a format conforming to the proofreading function in that document editing application. For example, if the specific document editing application is Microsoft Word (trademark), the correction content corresponding to the proofreading symbol detected from the scanned image may be added to the electronic document as the "recording of change history (revision history)" in the proofreading function of the application.

[0084] In S818, the job control unit 403 transmits the electronic document corrected in S817 or S822 to the cloud storage via the cloud storage communication unit 405, and instructs the cloud storage to update the electronic document to be reflected. As a result, the electronic document to be reflected stored in the cloud storage is overwritten by the electronic document transmitted from the job control unit 403 (that is, the electronic document corrected in S817 or S822). In addition, when the cloud storage manages the versions of the target electronic document, the electronic document modified in the above overwriting process is treated as a new version of the electronic document to be reflected. As a specific example, when the process of S817 is executed, a new version of the electronic document in which the proofreading information is reflected in S817 is added to the electronic document to be reflected stored in the cloud storage. By performing such management, for example, it becomes possible to control to return the electronic document stored in the cloud storage to the version before the above overwriting process, or to present the modified part in the new version as a difference.

[0085] In S819, the job control unit 403 causes the data storage unit 402 to store the history related to the above series of job processes in a predetermined storage area. The information stored as the history includes, for example, information regarding the file name of the electronic document to be reflected, the processing date and time, and the success or failure of the processing. In this case, information indicating the success of the process related to the cloud scan is recorded as the history of the job process.

[0086] By applying the above-described processes, for example, even when the result of proofreading is handwritten and added to a printed copy of an electronic document, it is possible to automatically reflect the correction corresponding to the content indicated by the result of the proofreading on the electronic document stored in the cloud storage. Further, according to the information processing system according to the present embodiment, it is also possible to reflect the correction corresponding to the content indicated by the result of the proofreading on the above electronic document stored in the cloud storage as proofreading information.

[0087] <Modification Example> A modification example of the information processing system according to the present embodiment will be described below. In this modification example, an example will be described in which, when reflecting the correction corresponding to the content indicated by the result of proofreading on an electronic document as proofreading information, a clip image indicating the content of the proofreading added by handwriting or the like to a printed copy during proofreading is associated as a supplement. Note that in this modified example, the description will focus on the parts different from the above-described embodiment, and detailed descriptions of the parts substantially the same as the above-described embodiment will be omitted.

[0088] FIG. 9 is a flowchart showing an example of the processing of the information processing system according to this modified example, and shows an example of the processing related to the realization of the cloud scan function. A series of processing shown in FIG. 9 is started, for example, when the button 704 on the cloud scan screen 701 shown in FIG. 7 is pressed.

[0089] In S901, the operation control unit 301 registers the processing content of the cloud scan with the job control unit 303 according to the selection state of the cloud storage in the display area 702 of the cloud scan screen 701 and the input states of the check boxes 703 and 706. The job control unit 303 requests the reading processing unit 306 to execute the processing related to the reading of the document placed on the scanner unit 104 according to the registered processing content. In response to this request, the reading processing unit 306 controls the operation of the scanner unit 104 to execute the processing related to the reading of the document placed on the reading surface of the scanner unit 104.

[0090] In S902, the MFP server communication unit 310 transmits the data corresponding to the reading result of the document in S901 (that is, the data of the scanned image) to the MFP server 120 as a processing request. At this time, the information about the cloud storage selected in the display area 702 and the input states of the check boxes 703 and 706 are also notified to the MFP server 120. Then, the MFP server communication unit 310 determines whether the transmission of the data corresponding to the reading result of the document to the MFP server 120 is successful. When the MFP server communication unit 310 determines in S902 that the transmission of data according to the reading result of the document to the MFP server 120 has failed, the process proceeds to S922. In S922, the operation control unit 301 of the MFP 100 causes information regarding an error that has occurred in the display area 705 of the cloud scan screen 701 to be displayed. Thereafter, the series of processes shown in FIG. 9 ends. On the other hand, when the MFP server communication unit 310 determines in S902 that the transmission of data according to the reading result of the above document to the MFP server 120 has been successful, the process proceeds to S903.

[0091] In S903, the MFP communication unit 406 receives a processing request transmitted from the MFP 100 and registers the processing content corresponding to the processing request with the job control unit 403. The job control unit 403 determines whether the destination of the data corresponding to the reading result of the document has been specified as a cloud storage based on the information registered as an attribute in the processing request, or whether the association of URL information with the data has been specified.

[0092] When the job control unit 403 determines in S903 that the association of URL information with the data corresponding to the reading result of the document has not been specified and the destination cloud storage has been specified, the process proceeds to S925. In S925, the job control unit 403 transmits the data corresponding to the reading result of the document (in other words, the data of the scanned image) to the cloud storage specified as the above destination via the cloud storage communication unit 405.

[0093] On the other hand, when the job control unit 403 determines in S903 that the association of URL information with the data corresponding to the reading result of the document has been specified, the process proceeds to S904.

[0094] In S904, the job control unit 403 causes the image processing unit 404 to analyze the target data, and based on the result of the analysis, attempts to identify a URL for accessing a file (electronic document) on the cloud storage corresponding to the data. Then, the job control unit 403 determines whether the file on the cloud storage corresponding to the target data has been successfully acquired.

[0095] If the job control unit 403 determines in S904 that the file on the cloud storage corresponding to the target data could not be acquired, the process proceeds to S923. As a specific example, if access to the cloud storage fails, authentication fails, or the corresponding file does not exist on the cloud storage, the acquisition of the file targeted in S904 fails.

[0096] In S923, the job control unit 403 instructs the data storage unit 402 to store the history related to the job processing in a predetermined storage area. In this case, information indicating the failure of the process related to the cloud scan is recorded as the history related to the job processing. Then, the series of processes shown in FIG. 9 ends.

[0097] On the other hand, if the job control unit 403 determines in S904 that the file on the cloud storage corresponding to the target data has been successfully acquired, the process proceeds to S905. In S905, the job control unit 403 requests the image processing unit 404 to create a rendering image of the file (telecommunication document) acquired in S904. In response to this request, the image processing unit 404 generates a rendering image by performing rendering on the file.

[0098] In S906, the image processing unit 404 compares the scan image according to the result of the original document reading in S901 with the rendering image created in S905, and acquires a difference image according to the result of the comparison.

[0099] In S907, the image processing unit 404 recognizes the difference image acquired in S906 as a writing image indicating the writing performed on the printed matter of the electronic document that is the source of the rendering image. The image processing unit 404 performs predetermined image processing on the writing image to decompose the writing image into proofreading marks and natural language. Further, the image processing unit 404 recognizes the content of the correction instruction represented by the proofreading marks decomposed from the writing image and converts the natural language decomposed from the writing image into text information. The conversion of natural language into text information can be realized by, for example, OCR processing or the like. Note that the natural language writing is used, for example, in combination with proofreading marks and serves a supplementary role such as presenting correction candidates.

[0100] In S908, the image processing unit 404 specifies the coordinates of the proofreading marks on the rendering image.

[0101] In S909, the image processing unit 404 extracts the region corresponding to the coordinates specified in S908 from the rendering image.

[0102] In S910, the image processing unit 404 specifies the character string presented in the region on the rendering image extracted in S909.

[0103] In S911, the image processing unit 404 searches the electronic document to be reflected for the character string specified in S910.

[0104] In S912, the image processing unit 404 performs processing for controlling the display mode so that each of a series of candidates of the character string in the electronic document searched in S911 is displayed in a manner different from other character strings with respect to the target electronic document. Note that in this modification example, as in the above-described embodiment, the image processing unit 404 performs processing for highlighting the character string that becomes the above-described reflection candidate with respect to the target electronic document.

[0105] In S913, the image processing unit 404 generates a rendering image by performing rendering on an electronic document that has been processed so that the character string corresponding to each of the series of candidates retrieved in S912 is highlighted.

[0106] In S914, the image processing unit 404 compares the rendering image generated in S905 with the rendering image generated in S913. Then, based on the result of the comparison, the image processing unit 404 identifies the coordinates on the rendering image generated in S905 that correspond to the area on the rendering image generated in S913 where the highlighting process has been performed.

[0107] In S915, the image processing unit 404 determines whether the coordinates of the area where the highlighting process has been performed identified in S914 match the coordinates of the proofreading symbol identified in S908. Note that the size of the area where the highlighting process has been performed and the size of the area of the proofreading symbol do not necessarily match. Therefore, for example, the image processing unit 404 may determine that the coordinates match between these areas when the size of the partial area that overlaps with the other area in at least one of these areas is equal to or greater than a predetermined ratio (for example, more than half).

[0108] If the image processing unit 404 determines in S915 that the coordinates of the area where the highlighting process has been performed do not match the coordinates of the proofreading symbol, the process proceeds to S912. In this case, the image processing unit 404 identifies other candidates anew and executes the processes after S912 for the other candidates.

[0109] On the other hand, if the image processing unit 404 determines in S915 that the coordinates of the area where the highlighting process has been performed match the coordinates of the proofreading symbol, the process proceeds to S916.

[0110] In S916, the job control unit 403 determines whether the specified operation mode is the proofreading mode based on the input state of the check box 706 registered as the processing content of cloud scanning for the job.

[0111] If the job control unit 403 determines in S916 that the specified operation mode is not the proofreading mode, the process proceeds to S924. In S924, the image processing unit 404 corrects the character string in the electronic document specified as a reflection candidate in S912 according to the content indicated by the proofreading symbol.

[0112] On the other hand, if the job control unit 403 determines in S916 that the specified operation mode is the proofreading mode, the process proceeds to S917. In S917, the image processing unit 404 determines whether the recognition of the content of the correction instruction represented by the proofreading symbol decomposed from the writing image in S907 is successful. As a specific example, even if the image processing unit 404 performs image processing such as pattern matching on the partial image of the area recognized as the proofreading symbol in the writing image and it is still difficult to identify the corresponding proofreading symbol, it may be determined that the recognition of the content of the correction instruction fails. Also, even if the image processing unit 404 performs image processing such as OCR processing on the partial image of the area recognized as the natural language in the writing image and it is still difficult to identify the corresponding character string, it may be determined that the recognition of the content of the correction instruction fails. Note that if the image processing unit 404 succeeds in identifying the character string, even if the character string is mis-converted to a character string different from the actual one, the recognition of the content of the correction instruction may be regarded as successful.

[0113] If the image processing unit 404 determines in S917 that the recognition of the content of the correction instruction represented by the proofreading symbol decomposed from the writing image is successful, the process proceeds to S918. In S918, the image processing unit 404 adds correction information corresponding to the content indicated by the proofreading symbol as proofreading information in the electronic document to the character string in the electronic document specified as a reflection candidate in S912. Furthermore, in S919, the image processing unit 404 associates a partial image of a portion corresponding to an instruction related to proofreading in the written image (difference image) acquired in S907 with the proofreading information reflected in the electronic document in S918. As a specific example, the image processing unit 404 may extract a partial image of an area where there is writing around the proofreading symbol from the above-mentioned written image and associate it with the above-mentioned proofreading information.

[0114] On the other hand, if the image processing unit 404 determines that it has failed to recognize the content of the correction instruction represented by the proofreading symbol decomposed from the written image in S917, the process proceeds to S919. In this case, in S919, the image processing unit 404 generates proofreading information associated with a partial image of a portion corresponding to an instruction related to proofreading in the above-mentioned written image, and reflects the proofreading information on the character string in the electronic document specified as a reflection candidate in S912.

[0115] In S920, the job control unit 403 transmits the electronic document corrected in S918, S919, S919, or S924 to the cloud storage via the cloud storage communication unit 405. Furthermore, the job control unit 403 instructs the cloud storage to update the electronic document to be reflected. As a result, the electronic document to be reflected stored in the cloud storage is overwritten by the electronic document transmitted from the job control unit 403 (that is, the electronic document corrected in S918, S919, S919, or S924).

[0116] In S921, the job control unit 403 causes the data storage unit 402 to store the history related to the above series of job processes in a predetermined storage area. The information stored as the history includes, for example, information related to the file name, processing date and time, success or failure of processing, etc. of the electronic document to be reflected. In this case, information indicating the success of the processing related to cloud scanning is recorded as the history of the job processing.

[0117] By applying the above-described processing, for example, even when the result of proofreading is handwritten and added to a printed copy of an electronic document, it is possible to automatically reflect the correction corresponding to the content indicated by the result of the proofreading in the electronic document stored in the cloud storage. Also, according to the information processing system according to the present embodiment, it is also possible to reflect, as proofreading information, the correction corresponding to the content indicated by the result of the proofreading for the electronic document stored in the cloud storage. Also, in the information processing system according to this modification example, a clip image indicating an instruction corresponding to the proofreading result is associated with the proofreading information according to the reading result of a printed matter in which an instruction corresponding to the proofreading result is written by handwriting or the like. Therefore, even when an error occurs in recognizing the proofreading content written on a printed matter by image processing such as OCR processing, for example, the user can determine whether to reflect the correction while confirming the instruction corresponding to the proofreading result shown in the clip image.

[0118] <Other Embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a recording medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Description of Reference Numerals

[0119] 100 MFP 120 MFP Server 121 Cloud Storage 403 Job Control Unit 404 Image Processing Unit

Claims

A scanner unit that reads a printed matter with information indicating a location on a cloud storage where an electronic document of a printing source is stored attached as an image and generates a first image, and a first instruction for specifying whether to reflect a correction instruction written in the printed matter in the electronic document of the printing source stored at the location indicated by the information, a second instruction for specifying a cloud storage to which the first image is to be transmitted from a list of cloud storages available to the user, and a third instruction for specifying whether to add the correction instruction in a format conforming to a proofreading function of a document editing application capable of editing the electronic document of the printing source, an operation unit that displays a screen for receiving the above from the user, a receiving means for receiving the first image, the first instruction, the second instruction, and the third instruction from the device including the above, When it is specified in the first instruction received by the receiving means that the correction instruction is not reflected in the electronic document of the printing source, and a cloud storage to which the first image is to be transmitted is specified in the second instruction, a transmitting means for transmitting the first image received by the receiving means to the cloud storage specified in the second instruction, When it is specified in the first instruction received by the receiving means that the correction instruction is reflected in the electronic document of the printing source, and the electronic document of the printing source can be acquired from the location on the cloud storage based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, a second image is generated by rendering the electronic document of the printing source, and a difference image is generated by extracting a difference between the first image received by the receiving means and the generated second image, a generating means, By performing an analysis process and an OCR process on the difference image generated by the generating means, the correction instruction indicated by a proofreading symbol included in the difference image and text information in natural language included in the difference image are recognized, and further, based on the coordinates of the proofreading symbol, a specific means for specifying a character string in the electronic document to which the correction instruction is to be reflected, In the first instruction received by the receiving means, it is specified to reflect the correction instruction in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, the electronic document of the printing source can be acquired from the location on the cloud storage, and when it is specified in the third instruction to add the correction instruction in a format conforming to the proofreading function of the document editing application, the processing of adding the correction content based on the correction instruction as a record of the change history in the proofreading function of the document editing application to the character string in the electronic document specified by the specifying means is executed, while In the first instruction received by the receiving means, it is specified to reflect the correction instruction in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, the electronic document of the printing source can be acquired from the location on the cloud storage, and when it is specified in the third instruction not to add the correction instruction in a format conforming to the proofreading function of the document editing application, the execution means for executing the process of reflecting the correction content based on the correction instruction to the character string in the electronic document specified by the specifying means without recording it as a record of the change history in the proofreading function of the document editing application, An information processing apparatus characterized by comprising the same.

2. The storage means for storing information indicating that the reflection of the correction instruction in the electronic document of the printing source has failed when, in the first instruction received by the receiving means, it is specified to reflect the correction instruction in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, the electronic document of the printing source cannot be acquired from the location on the cloud storage. The information processing apparatus according to claim 1, further comprising the same.

3. The execution means is specified to reflect the correction instruction in the electronic document of the printing source in the first instruction received by the receiving means, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, the electronic document of the printing source can be acquired from the location on the cloud storage, and it is specified to add the correction instruction in a format conforming to the proofreading function of the document editing application in the third instruction. Further, when the correction content based on the correction instruction is deletion, the information processing apparatus according to claim 1, which executes a process of adding a description indicating deletion of a character string in the electronic document specified by the specifying means as a record of the change history in the proofreading function of the document editing application to the electronic document.

4. The execution means is specified to reflect the correction instruction in the electronic document of the printing source in the first instruction received by the receiving means, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received by the receiving means, the electronic document of the printing source can be acquired from the location on the cloud storage, and it is specified to add the correction instruction in a format conforming to the proofreading function of the document editing application in the third instruction. Further, when the correction content based on the correction instruction is correction to another character string, the information processing apparatus according to claim 1, which executes a process of adding a description indicating that the character string in the electronic document specified by the specifying means is corrected to the other character string as a record of the change history in the proofreading function of the document editing application to the electronic document.

5. The execution means is specified to reflect the correction instruction in the electronic document of the printing source in the first instruction received by the receiving means, and based on the information indicating the location on the cloud storage where the electronic document of the printing source, which is obtained by analyzing the first image received by the receiving means, is stored, the electronic document of the printing source can be acquired from the location on the cloud storage, and when it is specified in the third instruction to add the correction instruction in a format conforming to the proofreading function of the document editing application, further, a partial image of the portion corresponding to the correction instruction is extracted from the difference image, and the extracted partial image is added to the electronic document as a record of the change history in the proofreading function of the document editing application. The information processing apparatus according to any one of claims 1 to 4, wherein the processing is executed.

6. The information processing apparatus according to claim 1, wherein the electronic document is an electronic document in XML format.

7. The location where the electronic document of the printing source of the printed matter is stored is specified by a URL or a directory path described in a character string format or a barcode format in the first image. The information processing apparatus according to claim 1.

8. An information processing method executed by an information processing apparatus, a scanner unit that reads a printed matter to which information indicating a location on a cloud storage where an electronic document of a printing source is stored is attached as an image to generate a first image; a first instruction for specifying whether to reflect a correction instruction written in the printed matter in the electronic document of the printing source stored at the location indicated by the information; a second instruction for specifying a cloud storage serving as a transmission destination of the first image from a list of cloud storages available to the user; and a third instruction for specifying whether to add the correction instruction in a format conforming to the proofreading function of a document editing application capable of editing the electronic document of the printing source. A receiving step of receiving the first image, the first instruction, the second instruction, and the third instruction from a device including an operation unit that displays a screen for receiving from the user; When it is specified in the first instruction received in the receiving step that the correction instruction is not to be reflected in the electronic document of the printing source, and a cloud storage serving as the transmission destination of the first image is specified in the second instruction, a transmission step of transmitting the first image received in the receiving step to the cloud storage specified in the second instruction; When it is specified in the first instruction received in the receiving step that the correction instruction is to be reflected in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received in the receiving step, when the electronic document of the printing source can be acquired from the location on the cloud storage, a generation step of generating a second image by rendering the electronic document of the printing source, and generating a difference image by extracting the difference between the first image received in the receiving step and the generated second image; A specifying step of recognizing the correction instruction indicated by the proofreading symbol included in the difference image and the text information in natural language included in the difference image by performing an analysis process and an OCR process on the difference image generated in the generating step, and further specifying the character string in the electronic document to be the target of reflecting the correction instruction based on the coordinates of the proofreading symbol; When it is specified in the first instruction received in the receiving step that the correction instruction is to be reflected in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received in the receiving step, the electronic document of the printing source can be acquired from the location on the cloud storage, and it is specified in the third instruction that the correction instruction is to be added in a format conforming to the proofreading function of the document editing application, while executing a process of adding the correction content based on the correction instruction as a record of the change history in the proofreading function of the document editing application to the character string in the electronic document specified by the specifying step; In the first instruction received in the receiving step, it is specified to reflect the correction instruction in the electronic document of the printing source, and based on the information indicating the location on the cloud storage where the electronic document of the printing source is stored, which is obtained by analyzing the first image received in the receiving step, the electronic document of the printing source can be acquired from the location on the cloud storage, and when it is specified not to add the correction instruction in a format conforming to the proofreading function of the document editing application in the third instruction, an execution step of executing a process of reflecting the correction content based on the correction instruction on the character string in the electronic document specified by the specifying step without recording the change history in the proofreading function of the document editing application, An information processing method characterized by having the above.

9. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 7.

Images