Image processing apparatus and image processing method

Abstract

To easily perform an operation to an image processing apparatus including user authentication from a communication-connected portable communication terminal in a noncontact manner and in a state where information security is taken into consideration.SOLUTION: An image processing apparatus comprises: an authentication unit for performing user authentication; an apparatus information generation unit for generating apparatus information including identification information for identifying the image processing apparatus; a display unit for displaying the generated apparatus information; and a job control unit for receiving setting information related to a function setting of a copy job for reading and printing a set document from a communication terminal that has read the apparatus information via a network, and execute the copy job based on the received setting information.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an image processing apparatus and an image processing method that are capable of contactless operation by receiving instructions related to job settings via communication and that can authenticate a user who performs the operation. [Background technology]

[0002] There has been a demand for the ability to remotely configure an image processing device in advance. There is already known technology for an external mobile communication terminal connected to the image processing device via communication to receive operations for the image processing device and to notify information related to the status and operation of the image processing device. Due to reasons such as wanting to reduce the risk of infection from COVID-19 and other diseases, there is a growing demand for contactless operation of image processing devices such as multifunction peripherals, scanners, and printers that are operated by an unspecified number of people or a specified number of people. For example, mobile communication terminals such as smartphones and tablet terminals have become widespread, and in an environment where everyone in an office or the like where an image processing device is installed has a mobile communication terminal, it is technically possible to operate the image processing device contactlessly using a mobile communication terminal. In order to ensure the security of the data handled, image processing devices installed in an office are often connected to a private network such as an in-house intranet that is isolated from the Internet.

[0003] On the other hand, mobile communication devices are often used with applications that require an internet connection, such as SNS (Social Networking Service) and email. In such a network environment, if each employee needs to switch the network settings of their smartphone or other device from an Internet connection to an intranet connection each time they want to operate a multifunction printer installed in the office contactlessly, the operation becomes cumbersome. Furthermore, switching proxy settings and other settings also becomes necessary, which significantly affects the functionality of apps and smartphones other than contactless operation. There is a strong demand for contactless operation of image processing devices such as multifunction printers installed in the office while maintaining the network settings connected to the Internet.

[0004] For example, the following technique is known as a technique for notifying a user that a document remains in an image forming device. A user starts using the MFP by holding a mobile terminal over the MFP. The NFC communication unit of the MFP reads the user's email address and Bluetooth (registered trademark) pairing code stored in the mobile terminal, and periodically checks the Bluetooth connection status. When the user carrying the mobile terminal leaves the MFP and goes out of the Bluetooth communication range, causing the connection to be lost, the control unit of the MFP detects the document, and if the document remains, it sends a message to the email address read by the NFC communication unit to urge the user to collect the document (see, for example, Patent Document 1).

[0005] Furthermore, the following technology is known for solving the problem of an image forming apparatus being used by another user. This technology relates to a system in which a mobile information terminal and an image forming apparatus are recognized using short-range wireless communication, and then the mobile information terminal performs various functions on the image forming apparatus. When the image processing apparatus acquires authentication data from the mobile information terminal via short-range wireless communication, it performs user authentication based on the acquired authentication data. If the user authentication is successful, the image processing apparatus changes its operating mode to correspond to an application running on the mobile information terminal. Then, jobs sent from the changed mobile information terminal are given priority to be executed by the image forming apparatus (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2018-121098 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-005073 Summary of the Invention [Problem to be solved by the invention]

[0007] Many office image processing devices have user authentication capabilities to manage billing and ensure the security of the data they handle. A typical user authentication method involves identifying users using identification information consisting of a username and password. When this functionality is applied to contactless operation, identification information such as a username and password is transmitted from the user's mobile communication device to the image processing device. While this is not so difficult over an intranet, exchanging identification information while the mobile communication device is connected to the Internet, as described above, poses a risk of information leakage over the Internet. That is, when identification information, subsequent operational instructions, and responses to them are transmitted over the Internet, they are potentially intercepted and analyzed by third parties, potentially leading to identity theft. This increases the risk of information handled by and stored in the image processing device being leaked. When communicating with the image processing device via a server, the risk of information stored on the server also increases.

[0008] One way to avoid such risks is to first operate the image processing device directly to perform user authentication, and then use the mobile communication terminal to perform contactless operations. However, this does not provide a completely contactless operation. It is also possible for the image processing device to be equipped with an NFC (Near Field Communication) reader, and for the NFC reader to perform contactless user authentication using an IC card for each user before contactless operation. In this case, however, it is necessary to prepare an IC card for user authentication separate from the mobile communication terminal. It is conceivable that a mobile communication terminal could send operational instructions to an image processing device via short-range wireless communication such as NFC or Bluetooth without using the Internet. However, since the speed of short-range wireless communication is usually slower than that of Internet communication, it would take a long time to exchange a lot of information, such as operational instructions for starting a job, which could cause user dissatisfaction.

[0009] When an image processing device is connected to an external server, such as a server that provides services, via the Internet, and a user's mobile communication device communicates with the image processing device via that server, the following approach is possible. The authentication information (e.g., username and password) used by the user to log in to the server can also be used to ensure the security of communications between the server and the image processing device. For example, the authentication information can be used as an encryption key to encrypt data exchanged between the server and the image processing device. In this way, a certain level of information security can be guaranteed. However, it can also be said that encryption can be deciphered over time. This invention has been made in consideration of the above circumstances, and enables operations on an image processing device, including user authentication, to be easily performed from a communication-connected mobile communication terminal in a non-contact manner while taking information security into consideration. [Means for solving the problem]

[0010] The present invention provides an image processing device comprising: a device information generation unit that generates device information including device identification information; an authentication unit that performs contactless user authentication between a communication terminal using communication that does not go through the Internet or by importing image information; and a job control unit that provides the device information to the communication terminal that has been user authenticated by the authentication unit, using communication that does not go through the Internet or as image information, and causes the communication terminal to acquire the device information, and executes a job on a device corresponding to the device information based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet.

[0011] From a different perspective, the present invention provides an image processing method including the steps of: a step in which a control unit of an image processing device generates device information including device identification information; a step in which a contactless user authentication is performed between the control unit of the image processing device and a communication terminal using communication that does not go through the Internet or by importing image information; a step in which the device information is provided to the communication terminal for which user authentication has been performed, using communication that does not go through the Internet or as image information, and the communication terminal is caused to acquire the device information; and a step in which a job is executed on a device corresponding to the device information based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet. [Effects of the Invention]

[0012] The image processing device according to the present invention allows operations on the image processing device, including user authentication, to be easily performed from a mobile communication terminal connected for communication in a contactless manner while taking information security into consideration. The image processing method according to the present invention also provides the same effects. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram schematically showing the overall configuration of an image processing system according to an embodiment of the present invention. [Figure 2] 1 is a first flowchart illustrating an example of processing according to the first embodiment. [Figure 3] 10 is a second flowchart illustrating an example of processing according to the first embodiment. [Figure 4] 2 is an explanatory diagram showing an example of a home screen displayed on an operation unit of the multifunction peripheral shown in FIG. 1; [Figure 5] 1. FIG. 4 is an explanatory diagram showing an example of a screen for explaining the procedure of a non-contact operation displayed on the operation unit of the multifunction peripheral shown in FIG. [Figure 6] 1. FIG. 4 is an explanatory diagram showing an input screen for user authentication information in an application executed on the portable communication terminal shown in FIG. [Figure 7] 2 is an explanatory diagram showing an example of a basic screen in an application of the mobile communication terminal shown in FIG. 1. FIG. [Figure 8] 1. FIG. 4 is an explanatory diagram showing an example of a screen relating to function settings for a copy job in an application of the portable communication terminal shown in FIG. [Figure 9] 1. FIG. 4 is an explanatory diagram showing an example of a screen relating to function settings for a scan job in an application of the portable communication terminal shown in FIG. [Figure 10] 1. FIG. 4 is an explanatory diagram showing an example of an operation for reading identification information displayed on an operation unit of the multifunction peripheral in an application of the portable communication terminal shown in FIG. [Figure 11] 1. FIG. 4 is an explanatory diagram showing an example of a screen displayed on the operation unit of the multifunction peripheral shown in FIG. 1 during job execution. [Figure 12] 2 is an explanatory diagram showing an example of a screen of the multifunction peripheral displayed on an operation unit of the multifunction peripheral shown in FIG. 1; FIG. [Figure 13] 10 is a first flowchart illustrating an example of processing according to the second embodiment. [Figure 14] 10 is a second flowchart illustrating an example of processing according to the second embodiment. [Figure 15] 10 is a third flowchart illustrating an example of processing according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] The present invention will be described in more detail below with reference to the accompanying drawings. Note that the following description is given by way of example only in all respects and should not be construed as limiting the present invention. (Embodiment 1) 1 is a diagram schematically illustrating the overall configuration of an image processing system 100 according to Embodiment 1. The image processing system 100 includes a multifunction peripheral 10 as an image processing device, a mobile communication terminal 30, and a network service 50. It is also possible to provide an external authentication server 70 (shown by a chain line in FIG. 1) so that user authentication for the multifunction peripheral 10 is performed by the external authentication server 70 rather than the network service 50. The multifunction device 10 is communicably connected to a network service 50 via a network (NW) shown by a solid line in Fig. 1. The network (NW) performs communication via the Internet. The mobile communication terminal 30 is communicably connected to the network service 50 via the network (NW). The mobile communication terminal 30 is also configured to be able to acquire identification information, which will be described later, from the multifunction device 10 (see the dashed arrow in the figure). It is assumed that the user of the mobile communication terminal 30 has registered an account with the network service 50 and is in a state in which the network service 50 can be used. If an external authentication server 70 is provided, the mobile communication terminal 30 is also connected to be able to communicate with the external authentication server 70. The external authentication server 70 performs user authentication for use of the network service 50. If an external authentication server 70 is not provided, the network service 50 is responsible for the user authentication process.

[0015] The multifunction peripheral 10 is an output device that can scan, for example, a paper document as input and output it as a printed matter by forming an image on paper. The multifunction peripheral 10 is also an output device that can output electronic data in a predetermined format (for example, PDF (Portable Document Format)) based on the scanned data of the document. Note that the input is not limited to a paper document. For example, it may be electronic data input from an external storage medium or an external device.

[0016] The multifunction device 10 comprises a control unit 11, an operation unit 13, a communication unit 15, a document reading unit 17, and an image processing unit 19. The control unit 11 is composed of hardware resources centered around a processor and software resources of a program for controlling the multifunction device 10. The functions of the control unit 11 are realized when the processor executes the control program. The control unit 11 includes an authentication unit 22, a job control unit 23, and a device information generation unit 25. The job control unit 23 includes a non-contact operation unit 21. The device information generation unit 25 generates an identification information code R2 and a session code, which will be described later. The operation unit 13 accepts user operations and displays responses to the operations, the status of the multifunction device 10, and notifications to the user. The communication unit 15 is a communication circuit and communication control firmware that communicates with external devices via the network NW.

[0017] The document reading unit 17 reads a paper document. The image processing unit 19 performs processing related to the image read by the document reading unit 17 and processing related to printing the image. That is, it processes and prints image data received from the scanned document or an external device, and transmits the image data to the external device. The multifunction device 10 may also include a location information providing unit 29 that provides the location where the multifunction device 10 is installed. The location information providing unit may detect the location of the multifunction device 10 using a function such as GPS (Global Positioning System). However, it may also simply hold location information. For example, it may store location information of the multifunction device 10 measured using GPS by an external device in a referable manner.

[0018] The mobile communication terminal 30 comprises a terminal control unit 31, a terminal display unit 33, a terminal communication unit 35, and a setting storage unit 37. The terminal control unit 31 is composed of hardware resources centered around a processor and processing programs such as an OS (Operating System) and applications executed by the processor. The functions of the terminal control unit 31 are realized by the processor executing the processing programs. The terminal display unit 33 displays applications executed by the processor and screens that launch those applications. The terminal communication unit 35 is a communication circuit and communication control firmware that communicates with external devices via the network NW. The setting storage unit 37 is a storage area on a storage device such as RAM (Random Access Memory) that stores settings for jobs to be executed by the multifunction device 10.

[0019] The processes of the multifunction peripheral 10, the mobile communication terminal 30, and the network service 50 in this embodiment will be described below with reference to the flowcharts of FIGS. 2 and 3 and to FIGS. The control unit 11 of the multifunction device 10 performs processing as an authentication unit 22 and a job control unit 23 . When the multifunction device 10 is powered on, returns from the power saving mode, or transitions to the initial state without any operation for a predetermined period of time, the control unit 11 displays the home screen W2 shown in FIG. 4 on the operation unit 13 (step S10). The home screen W2 shown in Fig. 4 is an operation screen that accepts the user's job selection and selection of various functions. Eight job / function selection buttons B2 are arranged on the home screen W2 shown in Fig. 4. In addition, a display forward button B4 is arranged to switch the display to hidden job / function selection buttons. The job / function selection buttons B2 accept the user's selection of desired jobs, function displays, etc.

[0020] For example, when a tap operation on the [Simple Copy] button located at the left end of the top row of the job / function selection buttons B2 shown in FIG. 4 is received, the control unit 11 receives the selection of a copy job. Then, a screen (not shown) for receiving function settings related to the selected [Simple Copy] is displayed. The function setting items received on the function setting screen correspond to the copy job setting screen W28 of the contactless operation app described below. One of the job / function selection buttons B2 shown in FIG. 4 is a remote operation button B6. When a tap operation on the remote operation button B6 by the user is received, the control unit 11 transitions the home screen W2 to the contactless operation reception screen W4 shown in FIG. 5. The layout of the job / function selection buttons B2 shown in FIG. 4 can be set and changed by the user. The same applies to the hidden job / function selection buttons displayed by operating the display forward button B4.

[0021] When a tap operation on the remote operation button B6 is received on the home screen W2 shown in FIG. 4, the control unit 11 responds by transitioning the home screen W2 to a non-contact operation reception screen W4 shown in FIG. The contactless operation acceptance screen W4 displays an explanation of the operation procedure for performing remote operations (contactless operations) on the multifunction device 10. A [Close] button B8 is located in the upper right corner. When the [Close] button B8 is operated, the control unit 11 returns the contactless operation acceptance screen W4 to the home screen W2 and ends the acceptance of remote operations. The user can operate one of the other job / function selection buttons B2 located on the home screen W2 to select a job and use the operation unit 13 to set the functions of the selected job and execute the job. While the contactless operation acceptance screen W4 is displayed, job selection and function setting using the operation unit 13 cannot be performed.

[0022] As shown in Fig. 5, the explanation of the operation procedure displayed on the non-contact operation reception screen W4 uses illustrations or animations. The explanation is made in the order of items (1) to (3): (1) document placement, (2) job selection and function setting by operation using a mobile communication terminal, and (3) job execution. Item (3) in the explanation of the operating procedure displays the identification information code R2, which is identification information that distinguishes the multifunction peripheral 10 from other devices on the network NW. In the example shown in FIG. 5, the identification information is displayed by encoding it as a two-dimensional code. Note that the means for transmitting the identification information as image information is not limited to two-dimensional codes. For example, it is also possible to encode it using at least one of barcodes, symbols, and alphanumeric characters. The displayed image information, such as the two-dimensional code, can be read by a scanning device using a camera or laser light, and the encoded identification information can be transmitted to the mobile communication terminal 30. In this embodiment, the identification information code R2 is read by a camera included in the mobile communication terminal 30, allowing the mobile communication terminal 30 to obtain the identification information of the multifunction peripheral 10 without going through the Internet.

[0023] In addition to transmitting identification information using image information such as two-dimensional codes, it is also possible to transmit identification information without going through the Internet using short-range wireless communication such as NFC or Bluetooth, or medium-range wireless communication compliant with Direct Wi-Fi. 2 shows the procedure before the remote operation button B6 is operated, i.e., when the home screen W2 is displayed, an application for contactless operation of the mobile communication terminal 30 is started and user authentication with the multifunction device 10 is successful. According to this procedure, the user can start contactless operation without touching the operation unit 13 of the multifunction device 10 at all.

[0024] The user instructs the launch of an application installed in the mobile communication terminal 30 (see step S100 shown in FIG. 2). This application is used to set and start a job without contacting the multifunction device 10, and is hereinafter referred to as a contactless operation app. It is assumed that the contactless operation app is pre-installed in the mobile communication terminal 30. The terminal control unit 31 of the mobile communication terminal 30 accepts a user operation to instruct the launch of the contactless operation app, and launches the app (step S102). That is, the terminal control unit 31 executes a processing program related to the contactless operation app.

[0025] In this embodiment, when the terminal control unit 31 that executes the contactless operation app launches the contactless operation app, it first performs user authentication processing. This is processing to authenticate a user who has authority to operate the multifunction device 10. This is similar to the user authentication performed using the operation unit 13 of the multifunction device 10. As shown in FIG. 6(A), the terminal control unit 31 displays a user authentication screen W20 on the terminal display unit 33, which prompts the user to enter a username and password necessary to operate the multifunction device 10 (step S104). The user authentication screen W20 displays an authentication information input window W22 that accepts input of a username and password and alphanumeric input keys B20 superimposed on the basic app screen.

[0026] When the user inputs a user name and password into the authentication information input window W22 (step S106), the terminal control unit 31 generates a two-dimensional code based on the input user authentication information and displays it on the screen (step S108). Fig. 6(B) shows a screen (authentication information code display screen W24) on which the authentication information code R20 based on the input user authentication information is displayed on the terminal display unit 33. The user transmits the user authentication information from the mobile communication terminal 30 to the multifunction device 10 in a contactless manner and without using the Internet by having the camera (not shown in Fig. 1) or document reading unit 17 of the multifunction device 10 read the authentication information code R20 displayed on the terminal display unit 33. In addition to transferring user authentication information using such codes, it is also possible to transfer user authentication information directly without going through the Internet using short-range wireless communication such as NFC or Bluetooth or medium-range wireless communication compliant with Direct Wi-Fi.

[0027] In the above-described user authentication, the user inputs a username and password into the mobile communication terminal 30. However, as an alternative method, the mobile communication terminal 30 may perform biometric authentication of the user. Many recent mobile communication terminals 30 have a function for reading a biometric image of the user, such as a fingerprint or face, and performing authentication based on characteristics unique to the user. By linking this biometric authentication function with the user authentication process of the multifunction device 10, it is possible to eliminate the need for the user to input a username and password each time.

[0028] In this way, the multifunction device 10 acquires user authentication information from the mobile communication terminal 30 by any means such as the camera, the document reading unit 17, short-range wireless communication, or medium-range wireless communication without using the Internet, and performs user authentication processing as the authentication unit 22 (step S12).Then, the result of the user authentication (OK or NG) is displayed on the operation unit 13 in a manner that the user can recognize. Before accepting the operation of the remote operation button B6, that is, while the home screen W2 shown in Fig. 4 is displayed on the operation unit 13, user authentication using the mobile communication terminal 30 may be performed. If the result of the user authentication is OK, the control unit 11 switches the home screen W2 to the non-contact operation acceptance screen W4 shown in Fig. 5 at this point (step S14). Then, as shown in step S124 described later, the control unit 11 waits for an instruction to start execution of a job and data related to function settings for that job to be sent from the mobile communication terminal 30 via the Internet. While the non-contact operation acceptance screen W4 is displayed on the operation unit 13, the control unit 11 of the multifunction device 10 accepts only operation instructions from the mobile communication terminal 30. It does not accept operations by other users using the operation unit 13 or remote operations from devices other than the mobile communication terminal 30. This prevents conflicts of operation instructions to the multifunction device 10.

[0029] If user authentication is successful (result OK) in step S14 described above and the contactless operation acceptance screen W4 is displayed on the operation unit 13, the user will see the operation procedure displayed on the contactless operation acceptance screen W4 at that time. The user follows operation procedure (1) on the contactless operation reception screen W4 to place a paper document on the multifunction device 10. If the contactless operation app of the mobile communication terminal 30 has not yet been launched, the user follows operation procedure (2) to launch the contactless operation app, select a job, and set functions for the selected job.

[0030] When the user performs an operation to launch the contactless operation app, the terminal control unit 31 of the mobile communication terminal 30 causes the terminal display unit 33 to display the basic app screen W26 of the contactless operation app shown in Fig. 7. Then, the terminal control unit 31 accepts the user's job selection and function settings for the selected job on the contactless operation app (step S110). Then, the accepted job selection and job function settings are stored in the setting storage unit 37 (step S112).

[0031] 7 to 9 are explanatory diagrams showing operation screens of the contactless operation application. Each operation screen will be explained below. Fig. 7 shows an example of the basic screen of the contactless operation app. The basic app screen W26 shown in Fig. 7 includes a menu display area R22 and a favorites list display area R24. In the example shown in Fig. 7, a copy job selection button B22 and a scan job selection button B24 are arranged in the menu display area R22. The copy job selection button B22 accepts the selection of a copy job. When the copy job selection button B22 is tapped, the terminal control unit 31 determines that the user has selected a copy job. The terminal control unit 31 then switches the basic application screen W26 to the copy job setting screen W28 shown in Fig. 8(A) and accepts function settings related to the copy job and an instruction to start the copy job.

[0032] On the other hand, when the scan job selection button B24 is tapped, the terminal control unit 31 determines that a scan job has been selected by the user. Then, the terminal control unit 31 switches the basic application screen W26 to the scan job setting screen W32 shown in Fig. 9(A) and accepts function settings related to the scan job and an instruction to start the scan job. 7 is an area where the user can select frequently used items and display them in list form. The favorites list can be displayed in ascending or descending order using the sort button B26.

[0033] 8A and 8B show examples of copy job setting screens. The copy job setting screen W28 in Fig. 8A includes a function setting area R26 and a [Start] button B28. The function setting area R26 accepts settings for various function items related to copy jobs. The function setting area R26 shown in FIG. 8(A) accepts settings for items such as "Number of Copies," "Paper Tray," "Double-Sided Copying," "Rotate Back Side 180 Degrees," and "Color Mode." For example, when the function setting "Number of Copies" is selected by tapping, the terminal control unit 31 displays a modal M20 for accepting input of the function setting for the number of copies and a number of copies setting screen W30 consisting of kana / number input keys B32, superimposed on the copy job setting screen W28, as shown in FIG. 8(B). This is the number of copies setting screen W30.

[0034] On the number of copies setting screen W30, the terminal control unit 31 accepts an operation to set the number of copies in the modal M20 using the kana / number input keys B32. After the number of copies is set, if the "Confirm" button B30 is tapped, the terminal control unit 31 confirms the number of copies setting and returns the number of copies setting screen W30 to the copy job setting screen W28. If the "Cancel" button to the left of the "Confirm" button B30 is tapped, the terminal control unit 31 returns the number of copies setting screen W30 to the copy job setting screen W28 without confirming the number of copies setting.

[0035] 9A and 9B show examples of scan job setting screens. The scan job setting screen W32 in Fig. 9A includes a function setting area R28 and a [Start] button B34. The function setting area R28 accepts settings for various function items related to scan jobs. In FIG. 9(A), the function setting area R28 accepts settings for items such as "Enter Address," "Send to Self," "Scan Settings," "Image Orientation," and "Color Mode." FIG. 9(B) shows an example of an email-to-self setting screen W34 that is displayed when a tap operation on "Send to Self" is accepted. In the email-to-self setting screen W34 shown in FIG. 9(B), a modal M22 for accepting input of one's own email address and alphanumeric input keys B20 are displayed superimposed on the scan job setting screen W32.

[0036] On the email-to-myself settings screen W34, the terminal control unit 31 accepts the input operation of the email address to myself. After the email address to myself has been input, if the [OK] button B36 is tapped, the terminal control unit 31 will confirm the setting of the email address to myself and return the email-to-myself settings screen W34 to the original scan job settings screen W32. If the [Cancel] button to the left of the [OK] button B36 is tapped, the terminal control unit 31 will return the email-to-myself settings screen W34 to the scan job settings screen W32 without confirming the setting of the email address to myself.

[0037] The operation screens of the contactless operation application as shown in FIGS. 7 to 9 accept the same function settings as those accepted by the control unit 11 of the multifunction device 10 via the operation unit 13. The selection of a job is accepted on the basic application screen W26 shown in Fig. 7, and function settings for the selected job are accepted on the copy job setting screen W28 shown in Fig. 8(A) or the scan job setting screen W32 shown in Fig. 9(A). Once the settings for the job are complete, the user taps the [Start] key to instruct execution of the job (step S114).

[0038] When a tap operation on the [Start] button B28 shown in Fig. 8(A) or the [Start] button B34 shown in Fig. 9(A) is received, the terminal control unit 31 displays the identification information code reading screen W36 shown in Fig. 10(A) on the terminal display unit 33 (step S116 shown in Fig. 3). The identification information code reading screen W36 is a screen related to an operation to read the identification information code R2 displayed on the contactless operation reception screen W4 of the multifunction device 10 with the camera (not shown in Fig. 1) of the mobile communication terminal 30.

[0039] As shown in Fig. 10(A), the identification code reading screen W36 has a code reading frame R30 and a message R32 for reading the identification code R2 at a predetermined position, and displays an image read by the camera. The message R32 displayed at the initial stage instructs the user to use the camera to read the identification code R2 displayed on the operation unit of the multifunction device 10. When the user aligns the code reading frame R30 with the identification code R2 displayed on the operation unit of the multifunction device 10 (see Fig. 10(B)), the terminal control unit 31 executing the contactless operation app recognizes the identification code R2 and extracts the identification information embedded in the identification code R2 (step S118).

[0040] As a result, the identification information of the multifunction device 10 is sent directly from the multifunction device 10 to the mobile communication terminal 30 without going through the Internet. In addition to the exchange of identification information using such a code, the identification information of the multifunction device 10 may be directly exchanged without going through the Internet using short-range wireless communication such as NFC or Bluetooth or medium-range wireless communication compliant with Direct Wi-Fi. That is, when the above-described identification information code reading screen W36 is displayed, the identification information may be transmitted via short-range wireless communication or medium-range wireless communication instead. The terminal control unit 31, which has acquired the identification information of the multifunction device 10, transmits the identification information, the selected job, and the function settings stored in the setting storage unit 37 to the network service 50 (step S120). When transmitting to the network service 50, the terminal control unit 31 displays a message R34 indicating that data related to a non-contact operation is being transmitted, instead of the above-mentioned message R32 (see FIG. 10(B)). Furthermore, when the data transmission is complete, a message R36 indicating the transmission is complete is displayed (see FIG. 10(C)). An [OK] button is arranged in the message R36, and when the [OK] button is tapped, the terminal control unit 31 switches the display from the identification information code reading screen W36 to the application basic screen W26.

[0041] The network service 50 receives the selected job and function settings together with the identification information of the multifunction device 10 from the multifunction device 10, associates the data, and stores them in memory with a job ID for identifying and managing the job (step S122). The multifunction device 10 performs HTTPS long polling to the network service 50 to inquire whether job setting information corresponding to the identification information of the multifunction device 10 exists (step S16). If the job setting information corresponding to the identification information of the multifunction device 10 exists, the network service 50 establishes a connection with the multifunction device 10 and transmits the job setting information stored in the memory to the multifunction device 10 (step S124).

[0042] When the selected job and function settings for the new job are received from the network service 50, the control unit 11 as the job control unit 23 of the multifunction device 10 executes the job based on the received selected job and function settings (step S18). While the job is being executed, the control unit 11 causes the operation unit 13 to display a remote job execution screen W6 shown in Fig. 11. The remote job execution screen W6 includes a message R4 informing the user that a job is being executed and a job icon R6. When the job execution is completed, the control unit 11 transmits a job completion notification together with the identification information to the network service 50 (step S20). Furthermore, the remote job execution screen W6 is switched to a remote job completion screen W8 shown in Fig. 12 and displayed for a predetermined period of time. The remote job completion screen W8 displays a message R8 informing the user that the job has been completed and a job completion icon R10. When the network service 50 receives the job completion notification from the multifunction device 10, it deletes the data related to the job stored in the memory (step S126) and also transmits the job completion notification and the identification information of the multifunction device 10 to the mobile communication terminal 30 (step S128).

[0043] When the job completion notification is received from the network service 50, the terminal control unit 31 displays on the terminal display unit 33 that the job related to the multifunction device 10 has been completed, thereby notifying the user (step S130). The above is the flow of the processes executed by the multifunction peripheral 10, the portable communication terminal 30, and the network service 50 regarding contactless user authentication and contactless operation.

[0044] (Embodiment 2) The first embodiment is an example of a flow in which user authentication of the multifunction device 10 is performed immediately after the contactless operation app is launched. In contrast, in this embodiment, until the start of a job is instructed by the contactless operation app, identification information is acquired from the multifunction device 10 to prevent confusion with other jobs, but user authentication is not performed. Then, the contactless operation app accepts the selection of a job and the settings of the function items for that job. When the start of a job is instructed by the contactless operation app, user authentication is performed at that point. This allows another user to operate a contactless operation app on a mobile communication terminal other than the mobile communication terminal 30 in parallel and select a job and set functions for the multifunction device 10. Note that the job execution stage after user authentication is performed exclusively, so there is no confusion of jobs during execution.

[0045] 13 to 15 are flowcharts showing the processes of the multifunction peripheral 10, the mobile communication terminal 30, and the network service 50 according to this embodiment. Processes that are the same as or similar to those in FIGS. 2 and 3 according to the first embodiment are given the same reference numerals as those in FIGS. 2 and 3. The following mainly describes the processes that differ from those in FIGS. 2 and 3. 4, when a tap operation on the remote operation button B6 is received on the home screen W2 of the multifunction peripheral 10, the control unit 11, functioning as the device information generating unit 25, generates a code (session code) related to session information used to distinguish the multifunction peripheral 10 from other devices and to distinguish one job from another job. Then, the generated session code is displayed on the operation unit 13 (step S40).

[0046] In the first embodiment and in step S40 described above in this embodiment, the control unit 11 generates and displays a session code (see FIG. 5) when a tap operation on the remote operation button B6 on the home screen W2 of the multifunction peripheral 10 is accepted. Alternatively, the control unit 11 may generate a session code when the multifunction peripheral 10 is powered on or when the multifunction peripheral 10 returns from the energy-saving mode, and then display the contactless operation acceptance screen W4 on the operation unit 13 instead of the home screen W2. According to this embodiment, the user can start contactless operation without touching the operation unit 13 of the multifunction peripheral 10 at all. That is, the process of step S40 shown in FIG. 13 can be eliminated. Furthermore, the control unit 11 may receive a setting as to which screen, including the home screen W2 or the non-contact operation reception screen W4, is to be displayed on the operation unit 13 in the initial state. Then, the control unit 11 may display a screen according to the setting on the operation unit 13. The initial state may be when the power is turned on or when returning from energy saving mode, or when the [Reset] key (not shown) displayed on the screen of the operation unit 13 is operated, or when an auto-clear is activated after a predetermined period (for example, 60 seconds) has passed without any operation.

[0047] The user also starts the contactless operation app (step S100). When execution of the contactless operation app starts (step S102), the user uses the camera of the mobile communication terminal 30 to read the session code displayed on the operation unit 13. This operation causes the terminal control unit 31 to acquire session information specific to the multifunction device 10 that will execute the job for which an operation instruction is to be received and the job (step S140). This session information is used at least until user authentication is performed. After user authentication, the session information may also be used as identification information for the multifunction device 10. However, in the flowcharts shown in FIGS. 13 to 15, the identification information code for the multifunction device 10 is acquired separately along with the result of user authentication.

[0048] The terminal control unit 31 accepts job selection and function settings using a contactless application, as in the first embodiment. The terminal control unit 31 may be configured to transmit information related to the selected job and function settings together with session information to the network service 50 each time a job is selected or function settings are made (step S120 shown in a chain-line box C01 in FIG. 13). In this case, the network service 50 that receives the selected job and function settings associates the data, assigns a job ID for identifying and managing the job, and stores the data in memory. When new function settings or the like are received, the stored data is updated (step S122). Note that, unlike the first embodiment, when the procedure shown in the chain-line box C01 is configured to be executed, the data related to the selected job and function settings managed by the network service 50 includes an attribute indicating whether or not an instruction to start the job has been received. Unlike the configuration shown in the dashed line frame C01, the processing procedure may be configured so that the terminal control unit 31 transmits the selected job and function settings to the network service 50 all at once after user authentication, which will be described later.

[0049] When the settings related to the job are completed and a tap operation on the [START] button B28 shown in FIG. 8A or the [START] button B34 shown in FIG. 9A is received to start the job (step S114), the terminal control unit 31 performs a user authentication process. That is, the terminal control unit 31 displays a user authentication screen W20 on the terminal display unit 33, prompting the user to enter a username and password required to operate the multifunction peripheral 10 at that time (step S104), and receives the input of the username and password (step S106). Based on the received input, the terminal display unit 33 displays an authentication information code R20 (step S108 shown in FIG. 14). When the user causes the multifunction peripheral 10 to read the authentication information code R20, the control unit 11 of the multifunction peripheral 10 performs user authentication in the same manner as in the first embodiment (step S12).

[0050] If the user authentication is successful, the control unit 11 displays a screen corresponding to the non-contact operation reception screen W4 (see FIG. 5) in the first embodiment on the operation unit (step S14). However, in this embodiment, since the job selection and function setting have already been completed, it is not necessary to display (2) corresponding to that as an explanation of the operation procedure. It is sufficient to display the document set in (1) and the identification information code R2 in (3). As in the first embodiment, in addition to transferring identification information using such codes, it is also possible to transfer user authentication information directly without going through the Internet using short-range wireless communication or medium-range wireless communication.

[0051] If the user authentication is successful, the terminal control unit 31 causes the terminal display unit 33 to display a screen for reading the identification information code R2 displayed on the operation unit 13 of the multifunction device 10 (step S116). When the user causes the camera of the portable communication terminal 30 to read the identification information code R2, the portable communication terminal 30 acquires the identification information of the multifunction device 10 (step S118). Here, as shown in the dashed-line box C01 described above, each time a selected job and function settings are made, the data is sent to the network service 50, and if the network service 50 has saved that information, the process shown in the dashed-line box C02 in Fig. 14 is performed at this point. That is, when the user has the mobile communication terminal 30 read the identification information code R2 displayed on the operation unit 13, the terminal control unit 31 extracts the identification information from the identification information code R2. Then, the identification information and an instruction to start the job are sent to the network service 50 together with the session information (step S142 shown in the dashed-line box C02).

[0052] The network service 50 receives the identification information code and the job start instruction together with the session information from the multifunction device 10, and updates the attribute of the corresponding session information data to indicate that the job start instruction has been received (step S144 shown in the dashed box C02). The multifunction device 10 performs HTTPS long polling to the network service 50 to inquire whether there is data for which a job start instruction has been received that corresponds to the identification information of the multifunction device 10 (step S16 shown in a dashed box C02).

[0053] If job setting information corresponding to the identification information of the multifunction device 10 exists, the network service 50 establishes a connection with the multifunction device 10 and transmits the job setting information stored in the memory to the multifunction device 10 (step S124 shown in FIG. 15). Thereafter, the multifunction device 10 executes the job in the same manner as in the first embodiment. 15 shows a process flow corresponding to a case where the mobile communication terminal 30 is configured to transmit the selected job and function settings all at once to the network service 50 after user authentication. As in the first embodiment, the terminal control unit 31 stores in the setting storage unit 37 the operations related to the job selection and function settings that were accepted before user authentication.

[0054] If user authentication is successful, the user has the camera of the mobile communication terminal 30 read the identification information code R2 displayed on the operation unit 13 of the multifunction device 10 (step S116 shown in Figure 14), and the mobile communication terminal 30 obtains the identification information of the multifunction device 10 (step S118). When the identification information of the multifunction device 10 is acquired, the terminal control unit 31 transmits the identification information and the selected job and function settings stored in the setting storage unit 37 to the network service 50 (step S120 shown in a chained box C03 in FIG. 15). The network service 50 receives the identification information, selected job, and function settings from the multifunction device 10, associates the data, assigns a job ID for identifying and managing the job, and stores the data in memory (step S122 shown in the dashed box C03).

[0055] The multifunction device 10 performs HTTPS long polling to the network service 50 to inquire whether job setting information corresponding to the identification information of the multifunction device 10 exists (step S16 shown in a dashed box C03). If job setting information corresponding to the identification information of the multifunction device 10 exists, the network service 50 establishes a connection with the multifunction device 10 and transmits the job setting information stored in the memory to the multifunction device 10 (step S124). Thereafter, the multifunction device 10 executes the job in the same manner as in the first embodiment. The above is the flow of processing in this embodiment.

[0056] (Embodiment 3) In the first and second embodiments, the transmission of user authentication information from the mobile communication terminal 30 to the multifunction device 10 and the transmission of identification information from the multifunction device 10 to the mobile communication terminal 30 are performed by image information using a two-dimensional code. However, it has been mentioned that the information may be transmitted directly between the mobile communication terminal 30 and the multifunction device 10 without going through the Internet by using short-range wireless communication such as NFC or Bluetooth or medium-range wireless communication compliant with Direct Wi-Fi. When using long-distance wireless communication, communication can be established even with a mobile communication terminal 30 that is not located near the multifunction peripheral 10.

[0057] In this embodiment, the mobile communication terminal is assumed to be capable of detecting its current location using a function such as GPS, and the multifunction device 10 is assumed to be equipped with a location information providing unit 29 that provides the location where the device itself is installed. User authentication information and identification information are transmitted via medium-range wireless communication. The terminal control unit 31 transmits the user name and password entered during user authentication, as well as information on the current location of the portable communication terminal 30, to the multifunction device 10 via long-range wireless communication.

[0058] The control unit 11 of the multifunction device 10 uses the current location information received from the mobile communication terminal 30 along with the authentication information to determine whether or not the current location of the mobile communication terminal 30 is within a predetermined range from the device's own location provided by the location information providing unit 29. If the current location is not within the predetermined range from the device's own location, the result of user authentication is deemed NG even if the determination based on the received user name and password is OK. In other words, the user authentication cannot be passed unless the user performs the user authentication operation within a predetermined range from the multifunction device 10. By configuring it this way, even if a user name and password are stolen and someone attempts to impersonate you, remote user authentication can be prevented, thereby ensuring a certain level of security.

[0059] As mentioned above, (i) The image processing device according to the present invention is characterized by comprising: a device information generation unit that generates device information including device identification information; an authentication unit that performs contactless user authentication between a communication terminal and the communication terminal using communication that does not go through the Internet or by importing image information; and a job control unit that provides the device information to the communication terminal that has been user authenticated by the authentication unit, using communication that does not go through the Internet or as image information, and causes the communication terminal to acquire the device information, and executes a job on a device corresponding to the device information based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet.

[0060] In this invention, the communication unit performs communication via the Internet, and a specific embodiment thereof is realized by, for example, hardware resources of a communication circuit for a wired LAN or a wireless LAN, and firmware (software resources) that performs communication with the Internet via the communication circuit. The mobile communication terminal is a device that has a function of communicating with the image processing device via the Internet and a function of performing user authentication by communicating or providing image information without the Internet. Specific examples of such a device include a smartphone, a tablet terminal, and a PC.

[0061] The authentication unit performs contactless user authentication between the user and the mobile communication terminal. Specific examples of this include a function realized by a processor executing control software that performs processing as the authentication unit. The processor may be the same as the processor used for the contactless operation unit.

[0062] Furthermore, a job refers to a unit of a series of processes related to image processing executed by an image processing device. Specific examples of such a job include a scan job that performs a series of processes as a scanner, a print job that performs a series of processes as a printer, and a copy job that performs a series of processes as a copier. Other examples include a document filing job and a preview job. However, the types of jobs that an image processing device can execute depend on the configuration of the device.

[0063] Furthermore, the job control unit controls the execution of jobs, and is specifically realized, for example, by a processor executing control software that performs processing as the job control unit. The non-contact operation unit performs the same process as when a corresponding operation is performed on the operation unit of the image processing device, based on an operation instruction received from the mobile communication terminal. An image processing device refers to a device that performs image processing, and examples thereof include single-function devices such as scanners, printers, and facsimile machines, as well as multifunction devices that have multiple functions.

[0064] Further, preferred embodiments of the present invention will be described. (ii) The authentication unit may determine whether the communication terminal performing the communication is an authenticated terminal for which user authentication has been performed or an unauthenticated terminal for which user authentication has not been performed, and the job control unit may accept setting information related to the job execution from the authenticated terminal but not from the unauthenticated terminal. In this way, by accepting operation instructions only from authenticated terminals, it is possible to prevent unauthenticated users from operating the image processing device and executing jobs, just as in the case of operations using the operation unit of the image processing device, thereby ensuring security through user authentication.

[0065] (iii) The authentication unit determines whether the communication terminal performing the communication is an authenticated terminal for which user authentication has been performed or an unauthenticated terminal for which user authentication has not been performed, and the job control unit receives setting information related to the job execution from the communication terminal, and if the setting information relates to the start of a job, determines whether the communication terminal is the authenticated terminal or the unauthenticated terminal, and if the communication terminal is the unauthenticated terminal, has the authentication unit perform the user authentication and accepts the setting information related to the start of the job after being authenticated, and if the setting information does not relate to the start of a job, accepts the setting information even if the communication terminal is the unauthenticated terminal. In this way, operation instructions related to starting a job can be accepted from a mobile communication terminal, and operation instructions related to starting a job can be accepted only from mobile communication terminals whose users have been authenticated. It is possible to prevent users of unauthenticated mobile communication terminals from executing jobs, ensuring security through user authentication. On the other hand, by accepting operation instructions from unauthenticated terminals as long as they are not related to starting a job, it is possible to accept operations such as function settings of the image processing device and job-related settings without or before user authentication. Therefore, job selection and function settings using the contactless operation app can be accepted in parallel on different mobile communication terminals of different users.

[0066] (iv) The authentication unit may perform user authentication with the mobile communication terminal using at least one of short-range wireless communication and medium-range wireless communication with location information as communication that does not go through the Internet. In this way, operation instructions can be received from the user's mobile communication terminal via the Internet, and user authentication can be performed while taking information security into consideration using short-range wireless communication that does not go through the Internet or medium-range wireless communication with location information, thereby blocking unauthorized remote operations. In recent years, mobile communication terminals generally have functions for short-range wireless communication, medium-range wireless communication, and obtaining location information in addition to Internet communication, so these functions can be used to perform contactless user authentication and contactless operation in a manner that takes information security into consideration.

[0067] (v) The short-range wireless communication may be communication using NFC, Bluetooth, or Bluetooth LE. Many mobile communication terminals are equipped with such short-range wireless communication technology. In this way, contactless user authentication can be performed using short-range wireless communication in a manner that takes information security into consideration. (vi) The device may further include a location information providing unit that provides location information of the device itself, wherein the long-range wireless communication is communication via Direct Wi-Fi or Wi-Fi ad hoc mode, and the authentication unit may obtain the current location of the user's mobile communication terminal via the Internet or the long-range wireless communication, and perform user authentication when the mobile communication terminal is located within a predetermined range of the location information of the device itself provided by the location information providing unit. In this way, location information of the user's mobile communication terminal obtained by GPS (Global Positioning System) or the like can be acquired, and user authentication can be performed only when the device is within a predetermined range from the device's own location. Therefore, even with medium-range wireless communication, which has a wider communication range than short-range wireless communication, unauthorized remote operations can be blocked, and contactless user authentication can be performed in a manner that takes information security into consideration.

[0068] (vii) The user authentication using the image information may involve reading a two-dimensional code or a biometric image using a camera. This makes it possible to block unauthorized remote operations and realize contactless user authentication based on information captured by the camera, while taking information security into consideration. Examples of biometric images include fingerprints, irises, and facial images.

[0069] (viii) The job control unit may receive the setting information from the portable communication terminal via a server on which the user has registered an account. In this way, the user logs in to an account registered on the server to communicate via that server, so information security is further ensured by authentication related to logging in to the server in addition to user authentication of the image processing device.

[0070] (ix) One aspect of the present invention includes an image processing method including the steps of: a control unit of an image processing device generating device information including device identification information; performing contactless user authentication with a communication terminal using communication not via the Internet or by importing image information; providing the device information to the communication terminal for which user authentication has been performed using communication not via the Internet or as image information, and having the device corresponding to the device information execute a job based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet.

[0071] The aspects of the present invention also include combinations of any of the above-described aspects. In addition to the above-described embodiment, various modifications of the present invention are possible. These modifications should not be interpreted as not falling within the scope of the present invention. The present invention should include all modifications within the scope of the claims and their equivalents. [Explanation of symbols]

[0072] 10: Multifunction device, 11: Control unit, 13: Operation unit, 15: Communication unit, 17: Document reading unit, 19: Image processing unit, 21: Non-contact operation unit, 22: Authentication unit, 23: Job control unit, 25: Device information generation unit, 29: Location information providing unit, 30: Portable communication terminal, 31: Terminal control unit, 33: Terminal display unit, 35: Terminal communication unit, 37: Setting storage unit, 50: Network service, 70: External authentication server, 100: Image processing system B2: Job / function selection button, B4: Display advance button, B6: Remote operation button, B8: [Close] button, B20: Alphanumeric input keys, B22: Copy job selection button, B24: Scan job selection button, B26: Sort button, B28, B34: [Start] button, B30, B36: [Enter] button, B32: Kana / number input keys, M20: Modal, R2: Identification information code, R4, R8: Message, R6: Job icon, R10: Job completion icon, R20: Authentication information code, R22: Menu display area, R24: Favorites list display area, R26, R28: Function setting area, R30: Code reading frame, R32, R34, R36: Message, W2: Home screen, W20: User authentication screen, W22: Authentication information input window, W24: Authentication information code display screen, W4: Contactless operation reception screen, W6: Remote job execution screen, W8: Remote job completion screen, W26: Basic app screen, W28: Copy job setting screen, W30: Number of copies setting screen, W32: Scan job setting screen, W34: Email to self setting screen, W36: Identification code reading screen

Claims

1. An image processing device, an authentication unit that performs user authentication; a device information generating unit that generates device information including identification information that identifies the image processing device; a display unit that displays a screen showing a two-dimensional code related to the device information when the user authentication by the authentication unit is successful; a job control unit that receives setting information related to function settings of a copy job for reading and printing a document set in the image processing device from the communication terminal that reads the device information via a network, and executes the copy job based on the received setting information; An image processing device comprising:

2. the authentication unit performs contactless user authentication using communication that does not involve the Internet or by capturing image information; The image processing device according to claim 1 , wherein the authentication unit performs user authentication using at least one of short-range wireless communication and medium-range wireless communication using location information as the communication not via the Internet.

3. The image processing device according to claim 2 , wherein the short-range wireless communication is communication using NFC, Bluetooth, or Bluetooth LE.

4. a location information providing unit that provides location information of the image processing device; The medium-range wireless communication is communication via Direct Wi-Fi or Wi-Fi ad-hoc mode, The image processing device according to claim 2, wherein the authentication unit acquires the current location of the user's communication terminal via the Internet or the long-range wireless communication, and performs user authentication if the acquired current location is within a predetermined range from the location related to the location information.

5. 3. The image processing device according to claim 2, wherein the user authentication using the image information involves reading a two-dimensional code or a biometric image using a camera.

6. The image processing apparatus according to claim 1 , wherein the job control unit receives the setting information from the communication terminal via a server on which a user has registered an account.

Images