Image processing device and image processing method

Abstract

To easily perform operations for an image processing device, including user authentication from a communicatively connected mobile communication terminal, in a contactless manner while taking information security into consideration.SOLUTION: In an image processing system 100, a compound machine which is an image processing device includes: an authentication unit which performs user authentication, a device information generation unit which generates device information including identification information which identifies the image processing device; and a job control unit which receives setting information relating to function settings of a scan job for reading and transmitting a set document from the communication terminal which has read the device information via a network and executes the scan job on the basis of the received setting information. The setting information includes an email address to which the scan data is to be transmitted. A mobile communication terminal includes a display unit which displays a screen for displaying the device information generated when the user authentication by the authentication unit is successful.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus and an image processing method that are capable of contactless operation by receiving instructions related to job settings via communication and that can authenticate a user who performs the operation. [Background technology]

[0002] There has been a demand for a way to configure an image processing device in advance from a remote location. There is already known a technology for receiving operations for the image processing device and notifying information related to the status and operations of the image processing device from an external mobile communication terminal connected to the image processing device via communication. Due to reasons such as wanting to reduce the risk of infection from the novel coronavirus and other diseases, there is an increasing demand for contactless operation of image processing devices such as multifunction peripherals, scanners, and printers that are operated by an unspecified or specified number of people. For example, mobile communication terminals such as smartphones and tablet terminals have become widespread, and in an environment where an image processing device is installed, anyone has a mobile communication terminal, it is technically possible to operate the image processing device in a non-contact manner using a mobile communication terminal. In order to ensure the security of the data handled, the image processing device installed in the office is often connected to a private network such as a company intranet that is isolated from the Internet.

[0003] On the other hand, mobile communication devices are often used for applications that require an Internet connection, such as SNS (Social Networking Service) and e-mail. In such a network environment, if each employee has to switch the network settings of the smartphone or the like from an Internet connection to an intranet connection each time to operate the multifunction printer installed in the office contactlessly, the operation is complicated. In addition, it is necessary to switch proxy settings, etc., which has a large impact on apps and smartphone functions other than contactless operation. There is a strong demand for contactless operation of image processing devices such as multifunction printers installed in the office while keeping the network settings connected to the Internet.

[0004] For example, the following technology is known as a technology for notifying a user that a document remains in an image forming device. A user starts using the MFP by holding a mobile terminal over the MFP. The NFC communication unit of the MFP reads the user's email address and Bluetooth (registered trademark) pairing code stored in the mobile terminal, and periodically checks the Bluetooth connection status. When the user holding the mobile terminal leaves the MFP and goes outside the Bluetooth communication range, causing the connection to be lost, the control unit of the MFP detects the document, and if the document remains, it sends a message to the email address read by the NFC communication unit to urge the user to collect it (for example, see Patent Document 1).

[0005] Furthermore, the following technology is known for solving the problem of an image forming apparatus being used by another user. This relates to a system in which a portable information terminal and an image forming apparatus are recognized using short-range wireless communication, and then the portable information terminal performs various functions on the image forming apparatus. When the image processing apparatus acquires authentication data from the portable information terminal via short-range wireless communication, it executes user authentication based on the acquired authentication data. If the user authentication is successful, the image processing apparatus changes to an operation mode corresponding to an application running on the portable information terminal. Then, jobs sent from the changed portable information terminal are preferentially executed by the image forming apparatus (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2018-121098 A [Patent Document 2] JP 2016-005073 A Summary of the Invention [Problem to be solved by the invention]

[0007] Many image processing devices installed in offices have a user authentication function to ensure the security of billing management and data handled. A typical means of user authentication is to identify a user using identification information consisting of a user name and a password. When this function is applied to non-contact operation, identification information such as a user name and a password is transmitted from the user's mobile communication terminal to the image processing device. If it is on an intranet, it is not so difficult, but if identification information is exchanged while the mobile communication terminal is connected to the Internet as described above, the risk of information leakage on the Internet path becomes a problem. That is, if the identification information, the subsequent operation instructions, and the responses thereto are transmitted and received on the Internet, the possibility of them being intercepted and analyzed by a third party and leading to impersonation increases. This increases the risk of information handled by the image processing device and information stored in the image processing device being leaked. If communication with the image processing device is performed via a server, the risk of information stored in the server being leaked also increases.

[0008] As a method for avoiding such a risk, for example, it is possible to directly operate the image processing device to perform user authentication, and then perform contactless operation using the mobile communication terminal, but this is not a completely contactless operation. It is also possible that the image processing device is equipped with an NFC (Near Field Communication) reader, and that the NFC reader performs contactless user authentication using an IC card for each user before performing contactless operations. In that case, however, it is necessary to prepare an IC card for user authentication in addition to the mobile communication terminal. It is possible for a mobile communication terminal to send operation instructions to an image processing device via short-range wireless communication such as NFC or Bluetooth without using the Internet. However, since the speed of short-range wireless communication is usually slower than that of Internet communication, it may take time to exchange a lot of information, such as operation instructions related to starting a job, which may cause user dissatisfaction.

[0009] When an image processing device is connected to an external server, for example, a server that provides a service, via the Internet, and a user's mobile communication terminal communicates with the image processing device via that server, the following measures can be considered. The authentication information (e.g., user name and password) that the user uses to log in to the server is also used to ensure the security of communications between the server and the image processing device. For example, the authentication information is used as an encryption key to encrypt data exchanged between the server and the image processing device. In this way, a certain level of information security is guaranteed. However, it can also be said that the encryption can be decrypted over time. This invention has been made in consideration of the above circumstances, and enables operations on an image processing device, including user authentication, to be performed easily from a communication-connected portable communication terminal in a non-contact manner while taking into consideration information security. [Means for solving the problem]

[0010] The present invention provides an image processing apparatus comprising: a device information generation unit that generates device information including device identification information; an authentication unit that performs contactless user authentication between a communication terminal using communication not via the Internet or by importing image information; and a job control unit that provides the device information to the communication terminal that has been user authenticated by the authentication unit, using communication not via the Internet or as image information, to allow the communication terminal to acquire the device information, and causes a device corresponding to the device information to execute a job based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet.

[0011] From a different perspective, the present invention provides an image processing method including the steps of: a control unit of an image processing device generating device information including device identification information; performing contactless user authentication with a communication terminal using communication not via the Internet or by importing image information; providing the device information to the communication terminal for which user authentication has been performed using communication not via the Internet or as image information to allow the communication terminal to acquire the device information; and causing an apparatus corresponding to the device information to execute a job based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet. Effect of the Invention

[0012] The image processing device according to the present invention enables operations for the image processing device, including user authentication, to be easily performed from a communicatively connected portable communication terminal in a contactless manner while taking information security into consideration. The image processing method according to the present invention also provides the same effects. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram illustrating an overall configuration of an image processing system according to an embodiment of the present invention. [Diagram 2] 4 is a first flowchart showing an example of a process according to the first embodiment. [Diagram 3] 11 is a second flowchart illustrating an example of the process according to the first embodiment. [Figure 4] 2 is an explanatory diagram showing an example of a home screen displayed on an operation unit of the multifunction peripheral shown in FIG. 1; [Diagram 5] 1. FIG. 4 is an explanatory diagram showing an example of a screen for explaining a non-contact operation procedure displayed on an operation unit of the multifunction peripheral shown in FIG. [Figure 6] 1. FIG. 4 is an explanatory diagram showing an input screen for user authentication information in an application executed on the portable communication terminal shown in FIG. [Figure 7] 2 is an explanatory diagram showing an example of a basic screen in an application of the portable communication terminal shown in FIG. [Figure 8] 1. FIG. 4 is an explanatory diagram showing an example of a screen relating to function settings for a copy job in an application of the portable communication terminal shown in FIG. [Figure 9] 1. FIG. 4 is an explanatory diagram showing an example of a screen relating to function settings for a scan job in an application of the portable communication terminal shown in FIG. [Figure 10] 1. FIG. 4 is an explanatory diagram showing an example of an operation for reading identification information displayed on an operation unit of the multifunction peripheral in an application of the portable communication terminal shown in FIG. [Figure 11] 1. FIG. 4 is an explanatory diagram showing an example of a screen displayed on an operation unit of the multifunction peripheral shown in FIG. 1 during job execution. [Figure 12] 2 is an explanatory diagram showing an example of a screen of the multifunction machine displayed on an operation unit of the multifunction machine shown in FIG. 1; [Figure 13] 11 is a first flowchart illustrating an example of a process according to the second embodiment. [Figure 14] 11 is a second flowchart illustrating an example of the process according to the second embodiment. [Figure 15] 13 is a third flowchart illustrating an example of a process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The present invention will be described in more detail below with reference to the drawings. Note that the following description is illustrative in all respects and should not be construed as limiting the present invention. (Embodiment 1) 1 is a diagram showing an outline of the overall configuration of an image processing system 100 according to embodiment 1. The image processing system 100 includes a multifunction device 10 as an image processing device, a portable communication terminal 30, and a network service 50. It is also possible to provide an external authentication server 70 (shown by a chain line in FIG. 1) and have the user authentication for the multifunction device 10 performed by the external authentication server 70 instead of the network service 50. The multifunction device 10 is communicatively connected to a network service 50 via a network (NW) shown by a solid line in Fig. 1. The network (NW) performs communication via the Internet. The portable communication terminal 30 is communicatively connected to the network service 50 via the network (NW). In addition, the portable communication terminal 30 is configured to be able to obtain identification information, which will be described later, from the multifunction device 10 (see the dashed arrow in the figure). It is assumed that the user of the mobile communication terminal 30 has registered an account with the network service 50 and is in a state in which the network service 50 can be used. If an external authentication server 70 is provided, the mobile communication terminal 30 is also connected so as to be able to communicate with the external authentication server 70. The external authentication server 70 performs user authentication relating to the use of the network service 50. If an external authentication server 70 is not provided, the network service 50 is responsible for the user authentication process.

[0015] The multifunction device 10 is an output device that can scan a paper document as an input, and output the scanned data as a printed matter by forming an image on paper. The multifunction device 10 is also an output device that can output electronic data in a predetermined format (e.g., PDF (Portable Document Format)) based on the scanned data of the document. Note that the input is not limited to a paper document. For example, electronic data input from an external storage medium or an external device may be used.

[0016] The multifunction device 10 includes a control unit 11, an operation unit 13, a communication unit 15, a document reading unit 17, and an image processing unit 19. The control unit 11 is composed of hardware resources, mainly a processor, and software resources of a program for controlling the multifunction device 10. The processor executes the control program to realize the functions of the control unit 11. The control unit 11 includes an authentication unit 22, a job control unit 23, and a device information generating unit 25. The job control unit 23 includes a non-contact operation unit 21. The device information generating unit 25 generates an identification information code R2 and a session code, which will be described later. The operation unit 13 accepts user operations and displays responses to the operations, the status of the multifunction device 10, and notifications to the user. The communication unit 15 is a communication circuit that communicates with external devices via the network NW, and firmware for communication control.

[0017] The document reading unit 17 reads a paper document. The image processing unit 19 performs processing related to the image read by the document reading unit 17 and processing related to printing the image. That is, the image processing unit 19 processes the read document or image data received from an external device, prints it, and transmits the image data to an external device. The multifunction device 10 may also include a location information providing unit 29 that provides the location where the multifunction device 10 is installed. The location information providing unit may detect the location of the multifunction device 10 using a function such as GPS (Global Positioning System). However, it may simply hold location information. For example, it may store location information of the multifunction device 10 measured by an external device using GPS in a referable manner.

[0018] The mobile communication terminal 30 includes a terminal control unit 31, a terminal display unit 33, a terminal communication unit 35, and a setting storage unit 37. The terminal control unit 31 is composed of hardware resources centered on a processor, and processing programs such as an OS (Operating System) and applications executed by the processor. The processor executes the processing programs to realize the functions of the terminal control unit 31. The terminal display unit 33 displays applications executed by the processor and screens for starting the applications. The terminal communication unit 35 is a communication circuit and communication control firmware that communicates with external devices via the network NW. The setting storage unit 37 is a storage area on a storage device such as a RAM (Random Access Memory) that stores settings for jobs to be executed by the multifunction device 10.

[0019] The processes of the multifunction device 10, the portable communication terminal 30 and the network service 50 in this embodiment will be described below with reference to the flowcharts of FIGS. 2 and 3 and to FIGS. The control unit 11 of the multifunction device 10 performs processing as an authentication unit 22 and a job control unit 23 . When the multifunction device 10 is powered on, returns from a power saving mode, or transitions to an initial state without any operation for a predetermined period of time, the control unit 11 displays the home screen W2 shown in FIG. 4 on the operation unit 13 (step S10). The home screen W2 shown in Fig. 4 is an operation screen that accepts the user's selection of jobs and selection of various functions. Eight job / function selection buttons B2 are arranged on the home screen W2 shown in Fig. 4. In addition, a display forward button B4 is arranged to switch the display to hidden job / function selection buttons. The job / function selection buttons B2 accept the selection of the user's desired job, function display, etc.

[0020] For example, when a tap operation of the [Simple Copy] button located at the left end of the upper row of the job / function selection buttons B2 shown in FIG. 4 is received, the control unit 11 receives the selection of a copy job. Then, a screen (not shown) for receiving function settings related to the selected [Simple Copy] is displayed. The function setting items received on the function setting screen correspond to a copy job setting screen W28 of the non-contact operation application described later. Also, one of the job / function selection buttons B2 shown in FIG. 4 is a remote operation button B6. When a tap operation of the remote operation button B6 by the user is received, the control unit 11 transitions the home screen W2 to a non-contact operation reception screen W4 shown in FIG. 5. The arrangement of the job / function selection buttons B2 shown in FIG. 4 can be set and changed by the user. The same applies to the hidden job / function selection buttons displayed by the operation of the display forward button B4.

[0021] When a tap operation on the remote operation button B6 is received on the home screen W2 shown in FIG. 4, the control unit 11 responds by transitioning the home screen W2 to a non-contact operation receiving screen W4 shown in FIG. The non-contact operation acceptance screen W4 displays an explanation of the operation procedure for performing a remote operation (non-contact operation) on the multifunction device 10. A [Close] button B8 is arranged in the upper right. When the [Close] button B8 is operated, the control unit 11 returns the non-contact operation acceptance screen W4 to the home screen W2 and ends the acceptance of the remote operation. The user can operate any of the other job / function selection buttons B2 arranged on the home screen W2 to select a job, and use the operation unit 13 to set the functions of the selected job to execute the job. While the non-contact operation acceptance screen W4 is displayed, job selection and function setting cannot be performed using the operation unit 13.

[0022] 5, the explanation of the operation procedure displayed on the non-contact operation reception screen W4 uses illustrations or animations. The operation procedure is explained in the order of items (1) to (3): (1) document placement, (2) job selection and function setting by operation using a mobile communication terminal, and (3) job execution. Item (3) in the explanation of the operation procedure displays the identification information code R2 related to the identification information that distinguishes the multifunction device 10 from other devices on the network NW. In the example shown in FIG. 5, the identification information is encoded into a two-dimensional code and displayed. Note that the means for transmitting the identification information as image information is not limited to the two-dimensional code. For example, it is also possible to encode the identification information using at least one of a barcode, a symbol, and an alphanumeric character. It is possible to transmit the encoded identification information to the mobile communication terminal 30 by having a camera or a scanning device using a laser beam read the displayed image information such as the two-dimensional code. In this embodiment, the identification information code R2 is read by a camera of the mobile communication terminal 30, so that the mobile communication terminal 30 can obtain the identification information of the multifunction device 10 without going through the Internet.

[0023] In addition to transmitting identification information using image information such as two-dimensional codes, it is also possible to transmit identification information without going through the Internet by using short-range wireless communication such as NFC or Bluetooth, or medium-range wireless communication compliant with Direct Wi-Fi. 2 shows the procedure before the remote operation button B6 is operated, i.e., when the home screen W2 is displayed, an application for performing contactless operation of the mobile communication terminal 30 is started and user authentication with the multifunction device 10 is successful. According to this procedure, the user can start contactless operation without touching the operation unit 13 of the multifunction device 10 at all.

[0024] The user instructs the launch of an application installed in the mobile communication terminal 30 (see step S100 shown in FIG. 2). This application is for setting and instructing the MFP 10 to start a job without contact, and is hereinafter referred to as a contactless operation app. It is assumed that the contactless operation app is pre-installed in the mobile communication terminal 30. The terminal control unit 31 of the mobile communication terminal 30 accepts a user's operation to instruct the launch of the contactless operation app, and launches the app (step S102). That is, the terminal control unit 31 executes a processing program related to the contactless operation app.

[0025] In this embodiment, when the terminal control unit 31 that executes the contactless operation app starts the contactless operation app, it first performs a user authentication process. This process authenticates a user who has the authority to operate the multifunction device 10. This is similar to the user authentication performed using the operation unit 13 of the multifunction device 10. As shown in FIG. 6(A), the terminal control unit 31 causes the terminal display unit 33 to display a user authentication screen W20 that prompts the user to enter a user name and password required to operate the multifunction device 10 (step S104). The user authentication screen W20 displays an authentication information input window W22 that accepts the input of a user name and password and alphanumeric input keys B20 superimposed on the basic app screen.

[0026] When the user inputs a user name and password into the authentication information input window W22 (step S106), the terminal control unit 31 generates a two-dimensional code based on the input user authentication information and displays it on the screen (step S108). Fig. 6(B) shows a screen (authentication information code display screen W24) on which the authentication information code R20 based on the input user authentication information is displayed on the terminal display unit 33. The user transmits the user authentication information from the mobile communication terminal 30 to the multifunction device 10 in a non-contact manner and without going through the Internet by having the camera (not shown in Fig. 1) or the document reading unit 17 of the multifunction device 10 read the authentication information code R20 displayed on the terminal display unit 33. In addition to transferring user authentication information using such a code, it is also possible to transfer user authentication information directly without going through the Internet using short-range wireless communication such as NFC or Bluetooth, or medium-range wireless communication compliant with Direct Wi-Fi.

[0027] In the above-mentioned user authentication, the user inputs a user name and a password into the portable communication terminal 30. Alternatively, the portable communication terminal 30 may perform biometric authentication of the user. Many of the recent portable communication terminals 30 have a function of reading a biometric image of the user, such as a fingerprint or face, and performing authentication based on characteristics unique to the user. By linking the biometric authentication function with the user authentication process of the multifunction device 10, it is possible to eliminate the need for the user to input a user name and password each time.

[0028] In this way, the multifunction device 10 acquires user authentication information from the portable communication terminal 30 by any of the means such as the camera, the document reading unit 17, short-range wireless communication or medium-range wireless communication without going through the Internet, and performs user authentication processing as the authentication unit 22 (step S12).Then, the operation unit 13 displays the result of the user authentication (OK or NG) in a manner that the user can recognize. Before accepting the operation of the remote operation button B6, that is, in a state in which the home screen W2 shown in Fig. 4 is displayed on the operation unit 13, user authentication may be performed using the mobile communication terminal 30. If the result of the user authentication is OK, the control unit 11 switches the home screen W2 to the non-contact operation acceptance screen W4 shown in Fig. 5 at this point (step S14). Then, as shown in step S124 described later, the control unit 11 waits for an instruction to start execution of a job and data related to function settings of the job to be sent from the mobile communication terminal 30 via the Internet. While the non-contact operation acceptance screen W4 is displayed on the operation unit 13, the control unit 11 of the multifunction device 10 accepts only operation instructions from the mobile communication terminal 30. It does not accept operations by other users using the operation unit 13 or remote operations from devices other than the mobile communication terminal 30. This prevents conflicts of operation instructions to the multifunction device 10.

[0029] If user authentication is successful (result OK) in step S14 described above and the contactless operation acceptance screen W4 is displayed on the operation unit 13, the user will see the operation procedure displayed on the contactless operation acceptance screen W4 at that time. The user follows the operation procedure (1) on the contactless operation reception screen W4 to place a paper document on the multifunction device 10. If the contactless operation app of the mobile communication terminal 30 has not yet been started, the user follows the operation procedure (2) to start the contactless operation app, select a job, and set functions for the selected job.

[0030] When the user performs an operation to start the contactless operation app, the terminal control unit 31 of the mobile communication terminal 30 causes the terminal display unit 33 to display the basic app screen W26 of the contactless operation app shown in Fig. 7. Then, the terminal control unit 31 accepts the user's job selection on the contactless operation app and function settings for the selected job (step S110). Then, the accepted job selection and job function settings are stored in the setting storage unit 37 (step S112).

[0031] 7 to 9 are explanatory diagrams showing operation screens of the non-contact operation application. Each operation screen will be described below. Fig. 7 is an example of a basic screen of a contactless operation application. The basic application screen W26 shown in Fig. 7 includes a menu display area R22 and a favorites list display area R24. In the example shown in Fig. 7, a copy job selection button B22 and a scan job selection button B24 are arranged in the menu display area R22. The copy job selection button B22 accepts the selection of a copy job. When a tap operation of the copy job selection button B22 is accepted, the terminal control unit 31 determines that a copy job has been selected by the user. Then, the terminal control unit 31 switches the application basic screen W26 to a copy job setting screen W28 shown in Fig. 8(A) and accepts function settings related to the copy job and an instruction to start the copy job.

[0032] On the other hand, when the scan job selection button B24 is tapped, the terminal control unit 31 determines that the user has selected a scan job. Then, the terminal control unit 31 switches the basic application screen W26 to a scan job setting screen W32 shown in Fig. 9(A) and accepts function settings related to the scan job and an instruction to start the scan job. 7 is an area where the user can select frequently used items and display them in list form. The favorites list can be switched between ascending and descending order display using the sort button B26.

[0033] 8A and 8B are examples of copy job setting screens. The copy job setting screen W28 in Fig. 8A includes a function setting area R26 and a [Start] button B28. The function setting area R26 accepts settings of various function items related to copy jobs. The function setting area R26 shown in FIG. 8(A) accepts settings for items such as [number of copies], [paper feed tray], [double-sided copy], [rotate the back side 180 degrees], and [color mode]. For example, when the function setting [number of copies] is selected by tapping, as the function setting, as shown in FIG. 8(B), the terminal control unit 31 displays a modal M20 for accepting input of the function setting for the number of copies, and a number of copies setting screen W30 consisting of kana / number input keys B32, superimposed on the copy job setting screen W28. This is the number of copies setting screen W30.

[0034] On the number of copies setting screen W30, the terminal control unit 31 accepts an operation to set the number of copies in the modal M20 using the kana / number input keys B32. After the number of copies is set, when the [OK] button B30 is tapped, the terminal control unit 31 confirms the setting of the number of copies and returns the number of copies setting screen W30 to the copy job setting screen W28. When the [Cancel] button to the left of the [OK] button B30 is tapped, the terminal control unit 31 returns the number of copies setting screen W30 to the copy job setting screen W28 without confirming the setting of the number of copies.

[0035] 9A and 9B are examples of a scan job setting screen. The scan job setting screen W32 in Fig. 9A includes a function setting area R28 and a [Start] button B34. The function setting area R28 accepts settings of various function items related to scan jobs. In FIG. 9(A), the function setting area R28 accepts settings for items such as [Enter address], [Send to self], [Reading settings], [Image orientation], and [Color mode]. FIG. 9(B) shows an example of a self-mail setting screen W34 that is displayed when a tap operation of [Send to self] is accepted. In the self-mail setting screen W34 shown in FIG. 9(B), a modal M22 that accepts input of a self-mail address and an alphanumeric input key B20 are displayed superimposed on the scan job setting screen W32.

[0036] On the email to self setting screen W34, the terminal control unit 31 accepts an input operation of an email address to self. After an email address to self is input, when the [OK] button B36 is tapped, the terminal control unit 31 confirms the setting of the email to self setting screen W34 and returns the email to self setting screen W34 to the original scan job setting screen W32. When the [Cancel] button to the left of the [OK] button B36 is tapped, the terminal control unit 31 returns the email to self setting screen W34 to the scan job setting screen W32 without confirming the setting of the email to self.

[0037] The operation screens of the contactless operation application as shown in FIGS. 7 to 9 receive function settings similar to those received by the control unit 11 of the multifunction device 10 via the operation unit 13. As shown in FIG. The selection of a job is accepted on the basic application screen W26 shown in Fig. 7, and function settings for the selected job are accepted on a copy job setting screen W28 shown in Fig. 8(A) or a scan job setting screen W32 shown in Fig. 9(A). Once the settings for the job are complete, the user taps the [Start] key to instruct execution of the job (step S114).

[0038] When a tap operation on the [START] button B28 shown in Fig. 8(A) or the [START] button B34 shown in Fig. 9(A) is received, the terminal control unit 31 displays the identification code reading screen W36 shown in Fig. 10(A) on the terminal display unit 33 (step S116 shown in Fig. 3). The identification code reading screen W36 is a screen related to an operation for reading the identification code R2 displayed on the non-contact operation reception screen W4 of the multifunction device 10 with the camera (not shown in Fig. 1) of the mobile communication terminal 30.

[0039] As shown in Fig. 10(A), the identification code reading screen W36 has a code reading frame R30 and a message R32 for reading the identification code R2 at a predetermined position, and displays an image read by the camera. The message R32 displayed at the initial stage instructs the user to use the camera to read the identification code R2 displayed on the operation unit of the multifunction device 10. When the user aligns the identification code R2 displayed on the operation unit of the multifunction device 10 with the code reading frame R30 (see Fig. 10(B)), the terminal control unit 31 executing the contactless operation app recognizes the identification code R2 and extracts the identification information embedded in the identification code R2 (step S118).

[0040] As a result, the identification information of the multifunction device 10 is sent directly from the multifunction device 10 to the portable communication terminal 30 without going through the Internet. In addition to the transfer of identification information using such a code, the identification information of the multifunction device 10 may be transferred directly without going through the Internet using short-range wireless communication such as NFC or Bluetooth, or medium-range wireless communication conforming to Direct Wi-Fi. That is, at the timing when the above-mentioned identification information code reading screen W36 is displayed, the identification information may be transmitted via short-range wireless communication or medium-range wireless communication instead. The terminal control unit 31, which has acquired the identification information of the multifunction device 10, transmits the identification information and the selected job and function settings stored in the setting storage unit 37 to the network service 50 (step S120). When transmitting to the network service 50, the terminal control unit 31 displays a message R34 indicating that data related to a non-contact operation is being transmitted, instead of the above-mentioned message R32 (see FIG. 10(B)). Furthermore, when the transmission of the data is completed, a message R36 indicating the completion of transmission is displayed (see FIG. 10(C)). An [OK] button is arranged in the message R36, and when the [OK] button is tapped, the terminal control unit 31 switches the display from the identification code reading screen W36 to the application basic screen W26.

[0041] The network service 50 receives the selected job and function settings together with the identification information of the multifunction device 10 from the multifunction device 10, associates the data, and stores them in memory with a job ID for identifying and managing the job (step S122). The multifunction device 10 performs HTTPS long polling on the network service 50 to inquire whether job setting information corresponding to the identification information of the multifunction device 10 exists (step S16). If there is job setting information corresponding to the identification information of the multifunction device 10, the network service 50 establishes a connection with the multifunction device 10 and transmits the job setting information stored in the memory to the multifunction device 10 (step S124).

[0042] When the selected job and function settings for the new job are received from the network service 50, the control unit 11 serving as the job control unit 23 of the multifunction device 10 executes the job based on the received selected job and function settings (step S18). During the execution of a job, the control unit 11 causes a remote job execution screen W6 shown in Fig. 11 to be displayed on the operation unit 13. On the remote job execution screen W6, a message R4 notifying the user that a job is being executed and a job icon R6 are arranged. When the job execution is completed, the control unit 11 transmits a job completion notification together with the identification information to the network service 50 (step S20). Furthermore, the remote job execution screen W6 is switched to a remote job completion screen W8 shown in Fig. 12 and displayed for a predetermined period of time. A message R8 notifying the user that the job has been completed and a job completion icon R10 are arranged on the remote job completion screen W8. When the network service 50 receives the job completion notification from the multifunction device 10, it deletes the data related to the job stored in the memory (step S126) and transmits the job completion notification and the identification information of the multifunction device 10 to the mobile communication terminal 30 (step S128).

[0043] When the job completion notification is received from the network service 50, the terminal control unit 31 causes the terminal display unit 33 to display that the job related to the multifunction device 10 has been completed, thereby notifying the user (step S130). The above is the flow of the processes executed by the multifunction device 10, the portable communication terminal 30, and the network service 50 with respect to contactless user authentication and contactless operation.

[0044] (Embodiment 2) The first embodiment is an example of a flow in which user authentication of the multifunction device 10 is performed immediately after the contactless operation app is started. In contrast, in this embodiment, until the start of a job is instructed by the contactless operation app, identification information is obtained from the multifunction device 10 to prevent confusion with other jobs, but user authentication is not performed. Then, the contactless operation app accepts the selection of a job and the settings of the function items of that job. When the start of a job is instructed by the contactless operation app, user authentication is performed at that point. This allows another user to operate the non-contact operation app in parallel on a mobile communication terminal other than the mobile communication terminal 30 and select a job and set functions for the multifunction device 10. Note that the job execution stage after user authentication is performed exclusively. Therefore, there is no confusion of jobs at the execution stage.

[0045] 13 to 15 are flowcharts showing the processes of the multifunction device 10, the mobile communication terminal 30, and the network service 50 according to this embodiment. Processes that are the same as or similar to those in Figs. 2 and 3 according to the first embodiment are given the same reference numerals as in Figs. 2 and 3. Processes that differ from those in Figs. 2 and 3 will mainly be described. 4, when a tap operation of the remote operation button B6 is received, the control unit 11 as the device information generating unit 25 generates a code (session code) related to session information used to distinguish the multifunction device 10 from other devices and to distinguish a job from other jobs. Then, the generated session code is displayed on the operation unit 13 (step S40).

[0046] In the first embodiment and in step S40 described above in this embodiment, the control unit 11 generates and displays a session code (see FIG. 5) when a tap operation of the remote operation button B6 on the home screen W2 of the multifunction device 10 is accepted. Alternatively, the control unit 11 may generate a session code and display the non-contact operation acceptance screen W4 on the operation unit 13 instead of the home screen W2 when the multifunction device 10 is turned on or when the multifunction device 10 returns from the energy saving mode. According to this aspect, the user can start a non-contact operation without touching the operation unit 13 of the multifunction device 10 at all. That is, the process of step S40 shown in FIG. 13 can be eliminated. The control unit 11 may also receive a setting as to which screen, including the home screen W2 or the non-contact operation reception screen W4, is to be displayed on the operation unit 13 in the initial state. Then, the control unit 11 may display a screen according to the setting on the operation unit 13. The initial state may include when the power is turned on or when returning from energy saving mode, or when the [Reset] key (not shown) displayed on the screen of the operation unit 13 is operated, or when an auto-clear is activated after a predetermined period of time (60 seconds in one example) has elapsed without any operation.

[0047] The user also starts the contactless operation app (step S100). When the execution of the contactless operation app starts (step S102), the user causes the camera of the mobile communication terminal 30 to read the session code displayed on the operation unit 13. This operation causes the terminal control unit 31 to obtain session information specific to the MFP 10 that will execute the job for which an operation instruction is to be received and the job (step S140). This session information is used at least until user authentication is performed. After user authentication, the session information may be used as identification information for the MFP 10, but in the flowcharts shown in FIGS. 13 to 15, an identification information code for the MFP 10 is obtained separately along with the result of user authentication.

[0048] The terminal control unit 31 accepts the selection of a job and the function setting using the contactless application in the same manner as in the first embodiment. The terminal control unit 31 may transmit information related to the selected job and the function setting together with the session information to the network service 50 each time a job is selected or a function setting is performed (step S120 shown in a chain line frame C01 in FIG. 13). In this case, the network service 50 that receives the selected job and the function setting associates the data, assigns a job ID for identifying and managing the job, and stores the data in memory. When new function settings or the like are received, the stored data is updated (step S122). Note that, unlike the first embodiment, when configured to execute the procedure shown in the chain line frame C01, the data related to the selected job and the function setting managed by the network service 50 includes an attribute indicating whether or not an instruction to start the job has been received. Unlike the configuration shown in the dashed line frame C01, the processing procedure may be configured so that the terminal control unit 31 transmits the selected job and function settings to the network service 50 all at once after user authentication, which will be described later.

[0049] When the settings related to the job are completed and a tap operation of the [START] button B28 shown in FIG. 8(A) or the [START] button B34 shown in FIG. 9(A) is accepted in order to start the job (step S114), the terminal control unit 31 performs a user authentication process. That is, the terminal display unit 33 displays a user authentication screen W20 that prompts the user to enter a user name and password required to operate the multifunction device 10 at that time (step S104), and accepts the input of the user name and password (step S106). The terminal display unit 33 displays an authentication information code R20 based on the accepted input (step S108 shown in FIG. 14). When the user causes the multifunction device 10 to read the authentication information code R20, the control unit 11 of the multifunction device 10 performs user authentication in the same manner as in the first embodiment (step S12).

[0050] If the user authentication result is OK, the control unit 11 causes the operation unit to display a screen corresponding to the non-contact operation reception screen W4 (see FIG. 5) in the first embodiment (step S14). However, in this embodiment, since the job selection and function setting have already been completed, it is not necessary to display (2) corresponding to them as an explanation of the operation procedure. It is sufficient to display the document set in (1) and the identification information code R2 in (3). As in the first embodiment, in addition to transmitting identification information using such a code, it is also possible to transmit and receive user authentication information directly without using the Internet by using short-range wireless communication or medium-range wireless communication.

[0051] If the user authentication is successful, the terminal control unit 31 causes the terminal display unit 33 to display a screen for reading the identification information code R2 displayed on the operation unit 13 of the multifunction device 10 (step S116). When the user causes the camera of the portable communication terminal 30 to read the identification information code R2, the portable communication terminal 30 acquires the identification information of the multifunction device 10 (step S118). Here, as shown in the dashed box C01 described above, each time a selected job and function settings are made, the data is sent to the network service 50, and if the network service 50 has saved that information, the process shown in the dashed box C02 in Fig. 14 is performed at this point. That is, when the user causes the mobile communication terminal 30 to read the identification information code R2 displayed on the operation unit 13, the terminal control unit 31 extracts the identification information from the identification information code R2. Then, the identification information and an instruction to start the job are sent to the network service 50 together with the session information (step S142 shown in the dashed box C02).

[0052] The network service 50 receives the identification information code and the job start instruction from the multifunction device 10 along with the session information, and updates the attributes of the corresponding session information data to indicate that the job start instruction has been received (step S144 shown in the dashed box C02). The multifunction device 10 performs HTTPS long polling on the network service 50 to inquire as to whether or not there is data for which a job start instruction has been received that corresponds to the identification information of the multifunction device 10 (Step S16 indicated in a dashed box C02).

[0053] If job setting information corresponding to the identification information of the multifunction device 10 exists, the network service 50 establishes a connection with the multifunction device 10 and transmits the job setting information stored in the memory to the multifunction device 10 (step S124 shown in FIG. 15). After that, the multifunction device 10 executes the job in the same manner as in the first embodiment. 15 indicates a process flow corresponding to a case in which the selected job and function settings are transmitted collectively from the mobile communication terminal 30 to the network service 50 after user authentication. As in the first embodiment, the terminal control unit 31 stores in the setting storage unit 37 operations related to job selection and function settings accepted before user authentication.

[0054] If user authentication is successful, the user has the camera of the portable communication terminal 30 read the identification information code R2 displayed on the operation unit 13 of the multifunction device 10 (step S116 shown in FIG. 14), and the portable communication terminal 30 obtains the identification information of the multifunction device 10 (step S118). When the identification information of the multifunction device 10 is acquired, the terminal control unit 31 transmits the identification information and the selected job and function settings stored in the setting storage unit 37 to the network service 50 (step S120 shown in a dashed box C03 in FIG. 15). The network service 50 receives the identification information, selected job, and function settings from the multifunction device 10, associates the data, assigns a job ID for identifying and managing the job, and stores the data in memory (step S122 shown in the dashed box C03).

[0055] The multifunction device 10 performs HTTPS long polling on the network service 50 to inquire as to whether or not job setting information corresponding to the identification information of the multifunction device 10 exists (Step S16 indicated by the dashed box C03). If job setting information corresponding to the identification information of the multifunction device 10 exists, the network service 50 establishes a connection with the multifunction device 10 and transmits the job setting information stored in the memory to the multifunction device 10 (step S124). Thereafter, the multifunction device 10 executes the job in the same manner as in the first embodiment. The above is the flow of processing in this embodiment.

[0056] (Embodiment 3) In the first and second embodiments, the user authentication information is transmitted from the mobile communication terminal 30 to the multifunction device 10 and the identification information is transmitted from the multifunction device 10 to the mobile communication terminal 30 by image information using a two-dimensional code. However, it has been described that the information may be transmitted directly between the mobile communication terminal 30 and the multifunction device 10 without going through the Internet by using short-range wireless communication such as NFC or Bluetooth or medium-range wireless communication conforming to Direct Wi-Fi. When using medium-range wireless communication, communication can be established even with a portable communication terminal 30 that is not located near the multifunction device 10.

[0057] In this embodiment, the mobile communication terminal is assumed to be capable of detecting the current location using a function such as GPS, and the multifunction device 10 is assumed to be equipped with a location information providing unit 29 that provides the location where the multifunction device is installed. User authentication information and identification information are transmitted via medium-range wireless communication. The terminal control unit 31 transmits information on the current location of the portable communication terminal 30, in addition to the user name and password entered during user authentication, to the multifunction device 10 via long-range wireless communication.

[0058] The control unit 11 of the multifunction device 10 uses the current location information received from the portable communication terminal 30 together with the authentication information for authentication judgment. Specifically, it judges whether the current location of the portable communication terminal 30 is within a predetermined range from the location of the device itself provided by the location information providing unit 29. If the current location is not within the predetermined range from the location of the device itself, the result of user authentication is deemed NG even if the judgment based on the received user name and password is OK. In other words, the user authentication cannot be passed unless the user performs the user authentication operation within a predetermined range from the multifunction device 10. By configuring it this way, even if a user name and password are stolen and someone spoofs their identity, remote user authentication can be eliminated, thereby ensuring a certain level of security.

[0059] As mentioned above, (i) An image processing device according to the present invention is characterized in that it comprises a device information generation unit which generates device information including device identification information; an authentication unit which performs contactless user authentication between a communication terminal using communication not via the Internet or by importing image information; and a job control unit which provides the device information to the communication terminal for which user authentication has been performed by the authentication unit, using communication not via the Internet or as image information, to cause the communication terminal to acquire the device information, and which causes a device corresponding to the device information to execute a job based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet.

[0060] In this invention, the communication unit performs communication via the Internet. A specific embodiment of the communication unit is realized by, for example, a hardware resource of a communication circuit of a wired LAN or a wireless LAN, and a firmware (software resource) that performs communication with the Internet via the communication circuit. The mobile communication terminal is a device having a function of communicating with the image processing device via the Internet and a function of performing user authentication by communicating without the Internet or by providing image information. Specific examples of the mobile communication terminal include a smartphone, a tablet terminal, and a PC.

[0061] The authentication unit performs contactless user authentication between the user and the mobile communication terminal. A specific example of the authentication unit is a function realized by a processor executing control software that performs processing as the authentication unit. The processor may be the same as the processor of the contactless operation unit.

[0062] A job refers to a unit of a series of processes related to image processing executed by an image processing device. Specific examples of the job include a scan job in which a series of processes are performed as a scanner, a print job in which a series of processes are performed as a printer, and a copy job in which a series of processes are performed as a copier. Other examples include a document filing job and a preview job. However, the types of jobs that an image processing device can execute depend on the configuration of the device.

[0063] Furthermore, the job control unit controls the execution of a job, and is realized, for example, by a processor executing control software that performs processing as the job control unit. Further, the non-contact operation section performs processing based on an operation instruction received from the portable communication terminal in the same manner as when a corresponding operation is performed on the operation unit of the image processing apparatus. The image processing device refers to a device that performs image processing, and examples of the image processing device include single-function devices such as scanners, printers, and facsimile machines, and multifunction devices that have multiple functions of these devices.

[0064] Further, preferred embodiments of the present invention will be described. (ii) The authentication unit may determine whether the communication terminal performing the communication is an authenticated terminal for which user authentication has been performed or an unauthenticated terminal for which user authentication has not been performed, and the job control unit may accept setting information related to the job execution from the authenticated terminal but not from the unauthenticated terminal. In this way, by accepting operation instructions only from authenticated terminals, it is possible to prevent unauthenticated users from operating the image processing device and executing jobs, just as in the case of operations using the image processing device's operation unit, thereby ensuring security through user authentication.

[0065] (iii) The authentication unit determines whether the communication terminal performing the communication is an authenticated terminal for which user authentication has been performed or an unauthenticated terminal for which user authentication has not been performed, and the job control unit receives setting information related to the job execution from the communication terminal, and if the setting information relates to the start of a job, determines whether the communication terminal is the authenticated terminal or the unauthenticated terminal, and if the communication terminal is the unauthenticated terminal, has the authentication unit perform the user authentication and accepts the setting information related to the start of the job after being authenticated, and if the setting information does not relate to the start of a job, accepts the setting information even if the communication terminal is the unauthenticated terminal. In this way, operation instructions related to starting a job can be accepted from a mobile communication terminal, and operation instructions related to starting a job can be accepted only from a mobile communication terminal that has been user authenticated. It is possible to prevent users of unauthenticated mobile communication terminals from executing jobs, and security is ensured by user authentication. On the other hand, by accepting operation instructions from unauthenticated terminals as long as they are not operation instructions related to starting a job, it is possible to accept operations for setting the functions of the image processing device and settings related to jobs without performing user authentication or at a stage before performing user authentication. Therefore, job selection and function setting by the non-contact operation app can be accepted in parallel on different mobile communication terminals of different users.

[0066] (iv) The authentication unit may perform user authentication between the mobile communication terminal and the mobile communication terminal using at least one of short-range wireless communication and medium-range wireless communication with location information as communication that does not go through the Internet. In this way, operation instructions can be received from the user's mobile communication terminal via the Internet, and user authentication can be performed while taking information security into consideration using short-range wireless communication that does not involve the Internet or medium-range wireless communication with location information. This makes it possible to block unauthorized remote operations. In recent years, mobile communication terminals generally have functions for short-range wireless communication, medium-range wireless communication, and obtaining location information in addition to Internet communication, so these functions can be used to perform contactless user authentication and contactless operation in a manner that takes information security into consideration.

[0067] (v) The short-range wireless communication may be communication using NFC, Bluetooth, or Bluetooth LE. Many mobile communication terminals are equipped with such short-range wireless communication capabilities. In this way, contactless user authentication can be performed using short-range wireless communication while taking information security into consideration. (vi) The device may further include a location information providing unit that provides location information of the device itself, and the medium-range wireless communication is communication via Direct Wi-Fi or Wi-Fi ad hoc mode, and the authentication unit may obtain a current location of the user's mobile communication terminal via the Internet or the medium-range wireless communication, and perform user authentication when the mobile communication terminal is located within a predetermined range of the location information of the device itself provided by the location information providing unit. In this way, location information of the user's mobile communication terminal obtained by a GPS (Global Positioning System) or the like can be obtained, and user authentication can be performed only when the device is within a predetermined range from the device's location. Therefore, even with medium-range wireless communication, which has a wider communication range than short-range wireless communication, unauthorized remote operations can be blocked, and contactless user authentication can be performed in a manner that takes information security into consideration.

[0068] (vii) The user authentication using the image information may involve reading a two-dimensional code or a biometric image using a camera. In this way, remote unauthorized operations can be blocked, and contactless user authentication based on information captured by the camera can be realized with consideration given to information security. Examples of biometric images include fingerprints, irises, and face images.

[0069] (viii) The job control unit may receive the setting information from the portable communication terminal via a server on which the user has registered an account. In this way, the user logs in to an account registered on the server to communicate via that server, so that information security is further ensured by authentication related to logging in to the server in addition to user authentication of the image processing device.

[0070] (ix) One aspect of the invention includes an image processing method comprising the steps of: a control unit of an image processing device generating device information including device identification information; performing contactless user authentication with a communication terminal using communication not via the Internet or by importing image information; providing the device information to the communication terminal for which user authentication has been performed using communication not via the Internet or as image information to allow the communication terminal to acquire the device information; and causing a device corresponding to the device information to execute a job based on the device information and setting information related to job execution transmitted from the communication terminal via the Internet.

[0071] The aspects of the present invention include any combination of the above-mentioned aspects. In addition to the above-mentioned embodiment, various modifications of the present invention are possible. These modifications should not be interpreted as not falling within the scope of the present invention. The present invention should include all modifications within the scope of the claims and the equivalent meanings. [Explanation of symbols]

[0072] 10: Multifunction device, 11: Control unit, 13: Operation unit, 15: Communication unit, 17: Document reading unit, 19: Image processing unit, 21: Non-contact operation unit, 22: Authentication unit, 23: Job control unit, 25: Device information generation unit, 29: Location information provision unit, 30: Portable communication terminal, 31: Terminal control unit, 33: Terminal display unit, 35: Terminal communication unit, 37: Setting storage unit, 50: Network service, 70: External authentication server, 100: Image processing system B2: Job / function selection button, B4: Display advance button, B6: Remote operation button, B8: [Close] button, B20: Alphanumeric input keys, B22: Copy job selection button, B24: Scan job selection button, B26: Sort button, B28, B34: [Start] button, B30, B36: [Enter] button, B32: Kana / number input keys, M20: Modal, R2: Identification code, R4, R8: Message, R6: Job icon, R10: Job completion icon, R20: Authentication code, R22: Menu display area, R24: Favorites list display area, R26, R28: Function settings area, R30: Code reading frame, R32, R34, R36: Message, W2: Home screen, W20: User authentication screen, W22: Authentication input window, W24: Authentication code display screen, W4: Contactless operation reception screen, W6: Remote job in progress screen, W8: Remote job completed screen, W26: Basic application screen, W28: Copy job setting screen, W30: Number of copies setting screen, W32: Scan job setting screen, W34: Email to self setting screen, W36: Identification code reading screen

Claims

1. An image processing device, an authentication unit for performing user authentication; a device information generating unit that generates device information including identification information for identifying the image processing device; a display unit that displays the generated device information; a job control unit that receives setting information related to function settings of a scan job for reading and transmitting a document set in the image processing device from the communication terminal that reads the device information via a network, and executes the scan job based on the received setting information; Equipped with The setting information includes an email address to which the scan data obtained by reading the document is to be sent, The display unit displays a screen for displaying the device information when the user authentication by the authentication unit is successful. Image processing device.

2. the authentication unit performs contactless user authentication by using communication not via the Internet or by capturing image information; The image processing device according to claim 1 , wherein the authentication unit performs user authentication using at least one of short-range wireless communication and medium-range wireless communication using location information as the communication not via the Internet.

3. The image processing device according to claim 2 , wherein the short-range wireless communication is communication according to NFC, Bluetooth, or Bluetooth LE.

4. a location information providing unit that provides location information of the image processing device, The medium-range wireless communication is communication by Direct Wi-Fi or Wi-Fi ad-hoc mode, The image processing device according to claim 2 , wherein the authentication unit acquires a current location of the user's communication terminal via the Internet or the long-range wireless communication, and performs user authentication if the acquired current location is within a predetermined range from a location related to the location information.

5. 3. The image processing apparatus according to claim 2, wherein the user authentication using the image information involves reading a two-dimensional code or a biometric image using a camera.

6. The image processing apparatus according to claim 1 , wherein the job control unit receives the setting information from the communication terminal via a server in which a user has registered an account.

Images