Processing system, information processing device, control program, and image processing device

Abstract

To provide a technique to improve convenience for a user who uses an image processing apparatus.SOLUTION: A processing system including an image processing apparatus comprising a display includes: an acquisition unit that acquires the degree of matching between a face image corresponding to registered face data and a picked-up image of the face of a user from a face authentication unit that performs face authentication processing based on the degree of matching; and a processing unit that, when the user is authenticated through the face authentication processing, displays, on the display, an execution instruction screen for receiving an instruction to execute image processing to be executed by the image processing apparatus. When the user is not authenticated through the face authentication processing, the processing unit displays, on the display, a different screen according to the level of the degree of matching.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a processing system including an image processing device having a display unit, an information processing device, a control program, and an image processing device. [Background technology]

[0002] Image processing devices such as multifunction peripherals may perform facial recognition before performing image processing such as printing or document scanning. The image processing device disclosed in Patent Document 1 photographs the face of a print user, extracts feature points from the photographed facial image and the facial image in the print information management table, and determines whether they are the same person. If the image processing device determines that the two people are different, it suspends processing, and if it determines that the two people are the same person, it allows printing of files associated with the print user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-15912 Summary of the Invention [Problem to be solved by the invention]

[0004] When facial recognition fails, users may not know why it failed and may not know what to do. [Means for solving the problem]

[0005] The processing system of the present invention is a processing system including an image processing device equipped with a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match. 、 the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; When the degree of match acquired by the acquisition unit is the second level, the processing unit displays on the display unit an imaging guide screen that prompts the user to perform imaging again to perform the face authentication process. , has an aspect. The processing system of the present invention is a processing system including an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; Let N be an integer greater than or equal to 2. The processing unit When the degree of match acquired by the acquisition unit for the user N consecutive times is at the second level, displaying on the display unit a user registration screen that prompts the user to generate the face data to be linked to user identification information that identifies the user; If the degree of match acquired by the acquisition unit is at the second level and the user registration screen is not displayed, an imaging guide screen that prompts the user to take another image to perform the face recognition process is displayed on the display unit. Furthermore, the processing system of the present invention is a processing system including an image processing device equipped with a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; The processing unit has a configuration in which, when the degree of match acquired by the acquisition unit is the first level, the processing unit displays a user registration screen on the display unit that prompts the user to generate the face data to be linked to user identification information that identifies the user.

[0006] Further, an information processing device of the present invention is an information processing device connected to an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match. 、 the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; When the degree of match acquired by the acquisition unit is the second level, the processing unit displays on the display unit an imaging guide screen that prompts the user to perform imaging again to perform the face authentication process. , has an aspect. Furthermore, the present invention provides an information processing apparatus connected to an image processing apparatus having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; Let N be an integer greater than or equal to 2. The processing unit When the degree of match acquired by the acquisition unit for the user N consecutive times is at the second level, displaying on the display unit a user registration screen that prompts the user to generate the face data to be linked to user identification information that identifies the user; If the degree of match acquired by the acquisition unit is at the second level and the user registration screen is not displayed, an imaging guide screen that prompts the user to take another image to perform the face recognition process is displayed on the display unit.

[0007] Furthermore, a control program of the present invention is a control program for controlling an image processing device having a display unit, an acquisition function for acquiring the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing function of displaying, on the display unit, an execution instruction screen for receiving an execution instruction for image processing to be executed by the image processing device when the user is authenticated by the face authentication processing, The processing function displays different screens on the display unit depending on the level of the degree of match when the user is not authenticated by the face authentication processing. 、 the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; When the degree of match acquired by the acquisition function is the second level, the processing function displays an imaging guide screen on the display unit to prompt the user to take an image for performing the face authentication process again. , has an aspect. Furthermore, a control program of the present invention is a control program for controlling an image processing device having a display unit, an acquisition function for acquiring the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing function of displaying, on the display unit, an execution instruction screen for receiving an execution instruction for image processing to be executed by the image processing device when the user is authenticated by the face authentication processing, the processing function displays different screens on the display unit depending on the level of the degree of match when the user is not authenticated by the face authentication processing; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; Let N be an integer greater than or equal to 2. The processing function is When the degree of match acquired by the acquisition function for the user N consecutive times is the second level, displaying a user registration screen on the display unit to prompt generation of the face data to be linked to user identification information that identifies the user; If the degree of match acquired by the acquisition function is at the second level and the user registration screen is not displayed, an imaging guide screen that prompts the user to take another image to perform the face recognition process is displayed on the display unit.

[0008] Furthermore, the image processing device of the present invention comprises: A display unit; an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an instruction to execute image processing, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match. 、 the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; When the degree of match acquired by the acquisition unit is the second level, the processing unit displays on the display unit an imaging guide screen that prompts the user to perform imaging again to perform the face authentication process. , has an aspect. Furthermore, the image processing device of the present invention comprises: A display unit; an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an instruction to execute image processing, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; Let N be an integer greater than or equal to 2. The processing unit When the degree of match acquired by the acquisition unit for the user N consecutive times is at the second level, displaying on the display unit a user registration screen that prompts the user to generate the face data to be linked to user identification information that identifies the user; If the degree of match acquired by the acquisition unit is at the second level and the user registration screen is not displayed, an imaging guide screen that prompts the user to take another image to perform the face recognition process is displayed on the display unit. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram schematically illustrating an example of the configuration of a system including a processing system. [Figure 2] FIG. 1 is a block diagram schematically illustrating an example of the configuration of an information processing device. [Figure 3] FIG. 1 is a block diagram schematically illustrating an example of the configuration of an image processing apparatus. [Figure 4] 10 is a flowchart schematically illustrating an example of a user registration process performed by a face authentication unit. [Figure 5] 10A and 10B are diagrams showing an example of calculating the degree of coincidence between a registered face image and a captured image. [Figure 6] FIG. 10 is a diagram schematically illustrating an example of the flow of processing performed in the processing system when a user is authenticated by face recognition processing. [Figure 7]FIG. 10 is a diagram schematically showing an example of a screen displayed on the display unit when a user is authenticated by face authentication processing. [Figure 8] 10 is a flowchart schematically illustrating an example of processing performed in a processing unit. [Figure 9] 10A and 10B are diagrams showing examples of screens that are displayed on the display unit according to the level of matching when a user is not authenticated by face authentication processing; [Figure 10] 10 is a flowchart schematically illustrating another example of processing performed in the processing unit. [Figure 11] FIG. 10 is a block diagram schematically illustrating another example configuration of a system including a processing system. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following describes embodiments of the present invention. Of course, the following embodiments are merely examples of the present invention, and not all of the features shown in the embodiments are necessarily essential to the solution of the invention.

[0011] (1) Overview of the technology included in this invention: First, an overview of the technology included in the present invention will be explained with reference to the examples shown in Figures 1 to 11. Note that the figures in this application are diagrams showing schematic examples, and the magnification in each direction shown in these figures may differ, and the figures may not be consistent with each other. Of course, each element of the present technology is not limited to the specific example indicated by the symbol. In the "Outline of the Technology Included in the Present Invention," the words in parentheses indicate supplementary explanations for the immediately preceding words.

[0012] Aspect 1: As illustrated in FIGS. 1 and 11 , a processing system SY1 according to one aspect of the present technology includes an image processing device 200 equipped with a display unit 206, and includes an acquisition unit U2 and a processing unit U3. The acquisition unit U2 acquires the degree of match C0 from a face authentication unit U1 that performs face authentication processing based on the degree of match C0 between a face image (e.g., registered face image IM3 shown in FIG. 5 ) corresponding to registered face data (e.g., registered face data DA2) and a captured image IM1 of a face F0 of a user US0. When the user US0 is authenticated by the face authentication processing, the processing unit U3 displays, on the display unit 206, an execution instruction screen (e.g., a function list screen D3 shown in FIG. 7 ) that accepts an execution instruction for image processing to be executed by the image processing device 200. When the user US0 is not authenticated by the face authentication processing, the processing unit U3 displays, on the display unit 206, a different screen (e.g., an image capture guide screen D4 or a user registration screen D5 shown in FIG. 9 ) depending on the level of the degree of match C0.

[0013] As described above, when user US0 is not authenticated by the facial recognition process, instead of a uniform screen, a different screen (D4 or D5) is displayed on the display unit 206 of the image processing device 200 depending on the level of the degree of match C0 between the registered face image IM3 and the captured image IM1. By looking at the screen (D4 or D5) on the display unit 206, the user US0 can understand the level of the degree of match C0 between the registered face image IM3 and the captured image IM1. Therefore, the above-described aspect 1 can provide a processing system SY1 that improves the convenience of user US0 who uses the image processing device 200.

[0014] Here, the image processing device 200 includes a multifunction device, a printer, a scanner, etc. The multifunction device means an image processing device 200 that has two or more functions among a plurality of functions including a printing function, a document reading function, a copying function, a facsimile function, etc. The image processing includes processes such as printing, document reading, document copying, and facsimile communication. The registered face data (DA2) may be data representing characteristic parts extracted from a face image (e.g., original face image IM2) used to register the face data (DA2), or may be the aforementioned face image (IM2) itself. The processing system SY1 may include a face authentication unit U1. The facial authentication unit U1 may generate authentication success information IN3 indicating that the user US0 has been authenticated through the facial authentication processing, and the acquisition unit U2 may acquire the authentication success information IN3. When the acquisition unit U2 acquires the authentication success information IN3, the processing unit U3 may display an execution instruction screen (D3) on the display unit 206. Furthermore, because it is possible to determine from the degree of match C0 whether the user US0 has been authenticated through the facial authentication processing, the processing unit U3 may display the execution instruction screen (D3) on the display unit 206 when it confirms that the facial authentication has been successful based on the degree of match C0 acquired by the acquisition unit U2. The level of the degree of coincidence C0 means an index that indicates the level of the degree of coincidence C0 when the degree of coincidence C0 is divided into a plurality of levels. The above remarks also apply to the following aspects.

[0015] Aspect 2: 8 to 10, the level of the degree of match C0 when the user US0 is not authenticated by the face authentication process may include a first level (e.g., a low level) and a second level (e.g., a medium level) higher than the first level. When the degree of match C0 acquired by the acquisition unit U2 is the first level, the processing unit U3 may display, on the display unit 206, a user registration screen D5 that prompts the generation of the face data (DA2) to be linked to user identification information UID that identifies the user US0. If the user US0 is not authenticated by the facial recognition process and the level of the degree of match C0 between the registered facial image IM3 and the captured image IM1 is a first level lower than a second level, a user registration screen D5 is displayed on the display unit 206 of the image processing device 200. By viewing the user registration screen D5, the user US0 can understand that they need to register as a user. Therefore, the above-described aspect 2 can further improve the convenience of the user US0 who uses the image processing device 200. Here, the level of matching when a user is not authenticated by facial recognition processing may include a third level that is different from the first and second levels. Note that "first," "second," ... in this application are terms for identifying each component included in multiple components having similarities, and do not indicate order. Furthermore, the level of matching when a user is authenticated by facial recognition processing may also include multiple levels. These remarks also apply to the following aspects.

[0016] Aspect 3: As illustrated in Figures 8 and 9, if the degree of matching C0 acquired by the acquisition unit U2 is the second level, the processing unit U3 may display an image capture guide screen D4 on the display unit 206 to prompt the user to capture an image again to perform the face recognition processing. If user US0 is not authenticated by the facial recognition process and the level of the degree of match C0 is a second level higher than the first level, an image capture guide screen D4 is displayed on the display unit 206 of the image processing device 200. By looking at the image capture guide screen D4, user US0 can understand that they should perform the operation to capture an image of face F0 again. Therefore, the above-mentioned aspect 3 can further improve the convenience of user US0 who uses the image processing device 200.

[0017] Aspect 4: 10 , when the degree of match C0 acquired by the acquisition unit U2 for the user US0 is the second level N consecutive times, where N is an integer equal to or greater than 2, the processing unit U3 may cause the display unit 206 to display a user registration screen D5 that prompts the user to generate the face data (DA2) to be linked to user identification information UID that identifies the user US0. When the degree of match C0 acquired by the acquisition unit U2 is the second level and the processing unit U3 does not display the user registration screen D5, the processing unit U3 may cause the display unit 206 to display an imaging guidance screen D4 that prompts the user to capture an image to perform the face authentication processing again. Even if the level of the degree of match C0 is a second level higher than the first level, if the degree of match C0 calculated in the face authentication process is repeatedly at the second level, the likelihood of successful face authentication is low. In this case, user registration screen D5 is displayed on display unit 206 of image processing device 200, and user US0 can see user registration screen D5 and understand that user registration is necessary. Therefore, aspect 4 described above can further improve the convenience of user US0 who uses image processing device 200.

[0018] Aspect 5: As shown in FIG. 1, the processing unit U3 may cause the display unit 206 to display a screen via a network NE1. The fifth aspect above can provide a suitable example of the processing system SY1 because it allows advanced processing to be performed by an information processing device (for example, the authentication printing server 100) connected to the image processing device 200 via a network.

[0019] Aspect 6: As illustrated in FIG. 11, the acquisition unit U2 and the processing unit U3 may be included in the image processing device 200. The sixth aspect can provide a processing system SY1 that does not require a server to manage the image processing device 200.

[0020] Aspect 7: 1 and 2, an information processing device (100) according to one aspect of the present technology is connected to an image processing device 200 having a display unit 206, and includes an acquisition unit U2 and a processing unit U3. The acquisition unit U2 acquires the degree of match C0 from a face authentication unit U1 that performs face authentication processing based on the degree of match C0 between a face image (IM3) corresponding to registered face data (DA2) and a captured image IM1 of a face F0 of a user US0. When the user US0 is authenticated by the face authentication processing, the processing unit U3 displays, on the display unit 206, an execution instruction screen (D3) that accepts an instruction to execute image processing to be executed by the image processing device 200. When the user US0 is not authenticated by the face authentication processing, the processing unit U3 displays, on the display unit 206, a different screen (D4 or D5) depending on the level of the degree of match C0. The seventh aspect can provide an information processing device (100) that improves the convenience of the user US0 who uses the image processing device 200.

[0021] Aspect 8: 2 and 11 , a control program PR1 according to one aspect of the present technology is a control program PR1 for controlling an image processing device 200 including a display unit 206, and causes a computer to implement an acquisition function FU2 and a processing function FU3. The acquisition function FU2 acquires the degree of match C0 from a face authentication unit U1 that performs face authentication processing based on the degree of match C0 between a face image (IM3) corresponding to registered face data (DA2) and a captured image IM1 of a face F0 of a user US0. When the user US0 is authenticated by the face authentication processing, the processing function FU3 displays, on the display unit 206, an execution instruction screen (D3) that accepts an execution instruction for image processing to be executed by the image processing device 200. When the user US0 is not authenticated by the face authentication processing, the processing function FU3 displays, on the display unit 206, a different screen (D4 or D5) depending on the level of the degree of match C0. The eighth aspect can provide a control program PR1 for improving the convenience of the user US0 who uses the image processing device 200.

[0022] Aspect 9: 11 , the image processing device 200 according to one aspect of the present technology includes a display unit 206, an acquisition unit U2, and a processing unit U3. The acquisition unit U2 acquires the degree of match C0 from a face authentication unit U1 that performs face authentication processing based on the degree of match C0 between a face image (IM3) corresponding to registered face data (DA2) and a captured image IM1 of the face F0 of the user US0. When the user US0 is authenticated by the face authentication processing, the processing unit U3 displays an execution instruction screen (D3) that accepts an instruction to execute image processing on the display unit 206. When the user US0 is not authenticated by the face authentication processing, the processing unit U3 displays a different screen (D4 or D5) on the display unit 206 depending on the level of the degree of match C0. The above-described aspect 9 can improve the convenience of the user US0 who uses the image processing device 200.

[0023] Furthermore, the present technology is applicable to a composite system including the processing system SY1, a composite device including the information processing device (100), an image processing system including the image processing device 200, a processing method performed in the processing system SY1, an information processing method performed in the information processing device (100), a processing method performed in the image processing device 200, a control method for the image processing device 200, a computer-readable medium on which the control program PR1 is recorded, etc. Any of the above-mentioned devices may be composed of multiple distributed parts.

[0024] (2) Specific examples of processing system configuration: FIG. 1 shows a schematic diagram of a system including a processing system SY1 as a specific example. The system includes an authentication printing server 100, an image processing device 200, a facial authentication server 400, an imaging device 500, and a terminal 600. Here, the authentication printing server 100 is an example of an information processing device, and the facial authentication server 400 is an example of a facial authentication unit U1. The processing system SY1 includes the authentication printing server 100 and the image processing device 200. The processing system SY1 may also include the terminal 600, the facial authentication server 400, or the imaging device 500. The authentication printing server 100, the image processing device 200, the facial authentication server 400, the imaging device 500, and the terminal 600 are connected to a network NE1 including the Internet. The network NE1 may include a LAN. Here, LAN is an abbreviation for Local Area Network. The connection to the network NE1 may be wired, wireless, or a combination of wired and wireless.

[0025] User US0 can register print job J0 to the authentication printing server 100 using a terminal 600 equipped with a CPU, ROM, RAM, a storage unit, an I / F for connecting to network NE1, and the like. Here, CPU is an abbreviation for Central Processing Unit, ROM is an abbreviation for Read Only Memory, RAM is an abbreviation for Random Access Memory, and I / F is an abbreviation for Interface. Examples of terminal 600 include computers such as personal computers including tablet terminals, and mobile phones such as smartphones. After registering print job J0 to the authentication printing server 100, user US0 can successfully authenticate his or her face F0 by pointing it toward the imaging device 500. Authenticated user US0 can also cause the image processing device 200 to execute printing based on the print job J0 registered in the authentication printing server 100. Furthermore, authenticated user US0 can cause the image processing device 200 to execute image processing such as copying, facsimile transmission, and document scanning. The combination of the image processing device 200 and the imaging device 500 is not limited to being one in the processing system SY1, and two or more may be present in the processing system SY1. Also, two or more terminals 600 may be present in the processing system SY1.

[0026] The authentication printing server 100 is a server computer including an acquisition unit U2 that acquires the degree of match C0 (described later) and a processing unit U3 that causes the image processing device 200 to perform image processing. The acquisition unit U2 can acquire a print job registration request RE1 from the terminal 600 and can acquire the degree of match C0 and authentication success information IN3 from the facial authentication server 400. When the print job registration request RE1 is received from the terminal 600, the processing unit U3 generates a print job J0 according to the print job registration request RE1 and stores the print job J0 in association with the user US0. The processing unit U3 also stores screen information D0 for displaying screens such as a pre-login screen D1, a facial authentication in-progress screen D2, a function list screen D3, an image capture guide screen D4, and a user registration screen D5 on the display unit 206 of the image processing device 200, as illustrated in FIGS. 7 and 9. The processing unit U3 can transmit screen information D0 appropriate for a given situation to the image processing device 200 and transmit the print job J0 associated with the user US0 to the image processing device 200. When the user US0 is authenticated by the face authentication process, the processing unit U3 displays a function list screen D3 on the display unit 206 of the image processing device 200. The function list screen D3 is an example of an execution instruction screen that accepts an instruction to execute image processing to be executed by the image processing device 200.

[0027] The image processing device 200 includes an input unit 205, a display unit 206, a printing unit 208, a card reader I / F 210, etc. The image processing device 200 may be a multifunction peripheral as shown in Fig. 3, or may be a dedicated printer, scanner, copier, facsimile, etc. A multifunction peripheral is a printing device that includes functions other than a printing function. Functions other than a printing function include a document reading function, a copying function, a facsimile function, etc. When the image processing device 200 receives screen information D0 from the authentication printing server 100 via the network NE1, it displays a screen according to the screen information D0 on the display unit 206. Furthermore, when the image processing device 200 receives a print job J0 from the authentication printing server 100 via the network NE1, it executes printing according to the print job J0 on the printing unit 208. Note that the image processing device 200 may acquire authentication information for authenticating the user US0 from a card reader and may transmit the acquired authentication information to the authentication printing server 100. Furthermore, the image processing device 200 may accept input of a user name and password on the input unit 205 and transmit the input user name and password to the authentication printing server 100 as authentication information.

[0028] The facial authentication server 400 is a server computer equipped with a CPU, ROM, RAM, a storage unit, an I / F for connecting to the network NE1, and the like. The storage unit of the facial authentication server 400 stores a face database DB2 in which registered face data DA2 based on an original facial image IM2 of the user US0 is associated with the user US0 and accumulated. Here, the original facial image IM2 is assumed to be a captured image obtained by capturing the face F0 of the user US0 using an imaging device 500 or the like for user registration, and may be a video. The registered face data DA2 is facial data representing characteristic features extracted from the original facial image IM2 and is face data registered in the face database DB2. The facial authentication server 400 receives the captured image IM1 from the imaging device 500 via the network NE1 and generates captured face data DA1 from the captured image IM1 for use in calculating the degree of match C0. The captured face data DA1 is facial data representing characteristic features extracted from the captured image IM1 and may be a video. The facial authentication process performed by the facial authentication server 400 will be described later.

[0029] 1 is a network camera that functions as a web server and includes a CPU, ROM, RAM, a storage unit, an imaging unit, an I / F for connecting to network NE1, and the like. Therefore, imaging device 500 can also be considered a server computer. When imaging device 500 captures a contactless image of the face of user US0, it transmits the captured image IM1 to face authentication server 400 via network NE1. It should be noted that imaging device 500 may be directly connected to face authentication server 400, rather than being a network camera.

[0030] 2 shows a schematic diagram of the configuration of an authentication printing server 100, an example of an information processing device. The authentication printing server 100 includes a processor, a ROM 102, a semiconductor memory, a RAM 103, a storage unit 104, an input device 105, a display device 106, a network I / F 107, and a clock circuit 108. These elements are electrically connected, allowing information to be input and output to and from each other.

[0031] The storage unit 104 stores an OS (not shown), a control program PR1, screen information D0, a print job J0, and the like. Here, OS is an abbreviation for operating system. The storage unit 104 is a computer-readable medium on which the control program PR1 is recorded. The control program PR1 may be recorded on an external computer-readable recording medium. The control program PR1 causes the authentication printing server 100 (as a computer) to realize an acquisition function FU2 for determining the degree of match C0, etc., and a processing function FU3 for causing the image processing device 200 to perform image processing. The acquisition function FU2 and the network I / F 107 constitute the acquisition unit U2 shown in FIG. 1. The processing function FU3 corresponds to the processing unit U3. The print job J0 is associated with the user US0 and stored in the storage unit 104. The example shown in Figure 2 indicates that the memory unit 104 stores a print job "J1" linked to a user "US1," a print job "J2" linked to a user "US2," and a print job "J3" linked to a user "US3." The storage unit 104 may be a magnetic storage device such as a hard disk, a nonvolatile semiconductor memory such as a flash memory, or the like.

[0032] The input device 105 may be a pointing device, hard keys including a keyboard, a touch panel attached to the surface of a display panel, or the like. The display device 106 may be a liquid crystal display panel, or the like. The network I / F 107 is connected to the network NE1 and communicates with other devices connected to this network NE1 in accordance with a predetermined communication standard. For example, the network I / F 107 receives a print job registration request RE1, a degree of match C0, authentication success information IN3, and the like from the other device, and transmits screen information D0, a print job J0, and the like to the image processing device 200, which is the other device. The clock circuit 108 is capable of outputting the current date and time.

[0033] The CPU 101 executes the control program PR1 read from the storage unit 104 to the RAM 103 to perform acquisition processing corresponding to the acquisition function FU2 and control processing corresponding to the processing function FU3. The control program PR1 causes the authentication printing server 100, which is a computer, to function as an acquisition unit U2 and a processing unit U3. The authentication printing server 100 executing the control program PR1 performs the acquisition process corresponding to the acquisition function FU2 and the control process corresponding to the processing function FU3.

[0034] 3 shows a schematic example of the configuration of an image processing device 200. The image processing device 200 includes a CPU 201, which is a processor, a ROM 202, which is a semiconductor memory, a RAM 203, which is also a semiconductor memory, a storage unit 204, an input unit 205, a display unit 206, a reading unit 207, a printing unit 208, a facsimile unit 209, a card reader I / F 210, a network I / F 211, and a clock circuit 212. These elements are electrically connected to each other, allowing information to be input and output to and from each other.

[0035] The storage unit 204 stores firmware and the like. The CPU 201 executes firmware read from the storage unit 204 to the RAM 203, thereby causing the image processing device 200 to realize multiple functions as the image processing device 200. These functions include a printing function, a reading function, a copying function, a facsimile function, and the like. The storage unit 204 may be a nonvolatile semiconductor memory such as a flash memory, a magnetic storage device such as a hard disk, or the like.

[0036] The input unit 205 may be a touch panel attached to the surface of a display panel, hard keys, or the like. The display unit 206 may be a liquid crystal display panel, or the like. The reading unit 207 reads an original document and generates scan data representing the read image. The printing unit 208 prints on printing paper based on the print job J0. The image processing device 200 performs a copy function by reading an original document with the reading unit 207 and printing the read image with the printing unit 208. The facsimile unit 209 facsimile-transmits the image read by the reading unit 207 to a destination via a telephone line, and receives an image from a sender via a telephone line. The image processing device 200 can print images received by the facsimile unit 209 with the printing unit 208. The network I / F 211 is connected to the network NE1 and communicates with the authentication printing server 100 connected to this network NE1 according to a predetermined communication protocol. For example, the network I / F 211 receives screen information D0, print job J0, etc. from the authentication print server 100. The clock circuit 212 can output the current date and time.

[0037] (3) Specific examples of processing performed by the face recognition unit: Fig. 4 shows a schematic example of the user registration process performed by the face authentication server 400, which is an example of the face authentication unit U1. Fig. 5 shows a schematic example of the process of calculating the degree of match C0 between a registered face image IM3 corresponding to the registered face data DA2 and a captured image IM1 of the face F0 of the user US0. Here, the registered face image IM3 refers to a face image reproduced from the registered face data DA2, and may differ from the original face image IM2 as shown in Fig. 5. The face authentication server 400 stores the registered face data DA2 in a storage unit in association with the user identification information UID by performing the processes of steps S102 to S108 shown in Fig. 4. Hereinafter, the notation of "step" may be omitted, and the step code may be shown in parentheses.

[0038] When the facial authentication server 400 receives an operation to start the user registration process, it starts the user registration process shown in Fig. 4. For example, the facial authentication server 400 displays a user registration start screen that accepts an operation to start the user registration process on the terminal 600 via the network NE1, and can then accept an operation to start the user registration process from the user US0 via the terminal 600. The facial authentication server 400 may also display a user registration start screen that accepts an operation to start the user registration process on the display unit 206 of the image processing device 200 via the authentication printing server 100. In this case, the facial authentication server 400 can accept an operation to start the user registration process from the user US0 via the image processing device 200 and the authentication printing server 100.

[0039] When the user registration process starts, the facial authentication server 400 acquires user identification information UID that identifies user US0 (S102). For example, the facial authentication server 400 may cause the terminal 600 to display an identification information input screen that accepts input of a user name and password via the network NE1, and may then accept an operation from user US0 to input the user name and password as the user identification information UID via the terminal 600. The facial authentication server 400 may also cause the display unit 206 of the image processing device 200 to display the identification information input screen via the authentication printing server 100. In this case, the facial authentication server 400 may accept an operation from user US0 to input the user name and password as the user identification information UID via the image processing device 200 and the authentication printing server 100. Furthermore, the image processing device 200 may acquire authentication information from a card reader as the user identification information UID and send it to the authentication printing server 100, so that the facial authentication server 400 receives the user identification information UID from the authentication printing server 100.

[0040] After acquiring the user identification information UID, the facial authentication server 400 acquires an original facial image IM2 of the user US0 (S104). For example, the facial authentication server 400 can receive the original facial image IM2 from the imaging device 500 via the network NE1 by having the imaging device 500 capture an image of the face F0 of the user US0 via the network NE1 immediately after the processing of S102. Alternatively, the facial authentication server 400 may receive the original facial image IM2 from the terminal 600 via the network NE1 by having the terminal 600 capture an image of the face F0 of the user US0 via the network NE1 immediately after the processing of S102.

[0041] After acquiring the original face image IM2, the face authentication server 400 generates registered face data DA2 based on the original face image IM2 (S106). FIG. 5 shows an example of generating registered face data DA2 from the original face image IM2. The face authentication server 400 detects a face portion from the original face image IM2, extracts feature portions from the face portion, such as a portion of the facial outline, the mouth, the nose, the left eye, and the right eye, and generates registered face data DA2 that indicates these feature portions with the coordinates of a predetermined number n of feature points P2i. Here, the predetermined number n is an integer greater than the number of feature portions. The variable i is a variable that identifies the feature points and is an integer from 1 to n. The coordinates of the feature points P2i can be expressed, for example, by an X coordinate in the horizontal direction and a Y coordinate in the vertical direction. The number of feature points P2i for each feature portion is predetermined and is not particularly limited, but can be, for example, approximately 5 to 100. The face portion can be detected by, for example, using a method for distinguishing between faces and non-faces using a support vector machine for the original face image IM2. The face portion can also be detected by using a method for distinguishing between faces and non-faces using a deep neural network, a general learning vector quantization method, or the like. The feature portion can be extracted by, for example, calculating the positional relationship of the feature portion or predetermined characteristics around the feature portion as feature amounts, and creating a feature amount vector. Examples of predetermined characteristics include color shading and color distribution. Of course, the feature portion can also be detected by other methods.

[0042] After the process of S106 shown in Fig. 4, the face authentication server 400 associates the registered face data DA2 with the user identification information UID and stores it in the face database DB2, stores the face database DB2 in the storage unit (S108), and ends the user registration process. The example shown in Fig. 4 shows that the storage unit 104 stores registered face data "DA21" associated with the user identification information "UID1," registered face data "DA22" associated with the user identification information "UID2," and registered face data "DA23" associated with the user identification information "UID3." It is assumed that the user identification information of the user "US1" shown in Fig. 2 is "UID1," the user identification information of the user "US2" shown in Fig. 2 is "UID2," and the user identification information of the user "US3" shown in Fig. 2 is "UID3."

[0043] The face authentication server 400 acquires a captured image IM1 from the imaging device 500, generates captured face data DA1 based on the captured image IM1, and calculates a degree of match C0 between the registered face image IM3 and the captured image IM1 based on the registered face data DA2 and the captured face data DA1. The degree of match is also called a degree of similarity. 5 shows an example of generating captured face data DA1 from a captured image IM1. The face authentication server 400 detects a face portion from the captured image IM1, extracts feature portions from the face portion, such as part of the facial contour, the mouth, the nose, the left eye, and the right eye, and generates captured face data DA1 that indicates these feature portions using the coordinates of the predetermined number n of feature points P1i. The coordinates of the feature points P2i can also be expressed, for example, using an X coordinate in the horizontal direction and a Y coordinate in the vertical direction. Each feature point P1i corresponds to a feature point P2i, and the number of feature points P1i in each feature portion is the same as the number of feature points P2i in each feature portion. The method for detecting a face part from the captured image IM1 can be a method for detecting a face part from the original face image IM2, and the method for extracting characteristic parts from a face part based on the captured image IM1 can be a method for extracting characteristic parts from a face part based on the original face image IM2.

[0044] Here, the coordinates of the feature point P1i are (X1i, Y1i), the coordinates of the feature point P2i are (X2i, Y2i), and the Euclidean distance between the feature points P1i and P2i is Li. The Euclidean distance Li is expressed by the following formula.

number

[0045] The face authentication server 400 performs a positioning process to align the captured face data DA1 with the registered face data DA2 so that the sum of squares of the Euclidean distances Li is minimized. If the sum of squares of the Euclidean distances Li is S, the sum of squares S is expressed by the following equation.

number

[0046] The alignment process can use at least one of a translation process for translating the captured face data DA1, a scaling process for enlarging or reducing the captured face data DA1, and a rotation process for rotating the captured face data DA1. Here, a reference feature point among the multiple feature points P2i is designated as P2r, and a reference feature point among the multiple feature points P1i is designated as P1r. The face recognition server 400 may translate the captured face data DA1 so that the feature point P1r aligns with the feature point P2r, and may perform the alignment process for the captured face data DA1 by performing at least one of a scaling process and a rotation process.

[0047] The face authentication server 400 calculates the degree of match C0 from, for example, the average value of the Euclidean distances Li after the alignment process. If the average value of the Euclidean distances Li, for example, the arithmetic mean value, is A, the average value A is expressed by the following formula.

number

[0048] The degree of agreement C0 is calculated so that it increases as the average value A decreases, for example. If a positive coefficient to be multiplied by the average value A is K, the degree of agreement C0 is expressed by the following formula, for example. C0=100-K×A …(4) The maximum value of the degree of match C0 calculated by equation (4) is 100. It can be said that the larger the degree of match C0, the closer the face F0 of user US0 is to the registered face image IM3, and the smaller the degree of match C0, the farther the face F0 of user US0 is from the registered face image IM3. Of course, the method for calculating the degree of match C0 is not limited to the method of calculation according to equation (4), and it can also be a method of calculation according to an equation different from equation (4), or a method in which a recognition unit using AI derives a score as the degree of match C0 based on the registered face data DA2 and the captured face data DA1. Here, AI is an abbreviation for Artificial Intelligence.

[0049] The facial authentication server 400 determines whether to authenticate the user US0 based on the calculated degree of match C0. For example, TH is a threshold that determines whether the degree of match C0 is high or low. If the degree of match C0 is equal to or greater than the threshold TH, the facial authentication server 400 generates authentication success information IN3 indicating that the user US0 has been authenticated, and transmits the authentication success information IN3, including information identifying the user US0, to the authentication printing server 100 along with the degree of match C0. If the degree of match C0 is less than the threshold TH, the facial authentication server 400 does not generate the authentication success information IN3, and transmits the degree of match C0 to the authentication server 400 without transmitting the authentication success information IN3. Of course, if the degree of match C0 is less than the threshold TH, the facial authentication server 400 may generate authentication failure information indicating that the user US0 was not authenticated, and transmit the authentication failure information, including information identifying the registered user for whom the degree of match C0 was calculated, as a candidate, to the authentication printing server 100 along with the degree of match C0. In this way, the face authentication server 400 performs face authentication processing based on the degree of coincidence C0 between the registered face image IM3 corresponding to the registered face data DA2 and the captured image IM1 of the face F0 of the user US0.

[0050] (4) Specific examples of system processing: FIG. 6 shows a schematic example of the processing flow performed by the processing system SY1 when user US0 is authenticated by facial recognition processing. Here, S202, S212, S220, and S226 correspond to the acquisition unit U2 and acquisition function FU2. S204, S206, S214, S222, and S228 correspond to the processing unit U3 and processing function FU3. Hereinafter, step symbols may be indicated in parentheses. FIG. 7 shows a schematic example of a screen displayed on the display unit 206 of the image processing device 200 when user US0 is authenticated by facial recognition processing.

[0051] First, the terminal 600 accepts an operation to register a print job J0 from the user US0, and transmits a registration request for the print job J0, i.e., a print job registration request RE1 shown in Fig. 1, to the authentication printing server 100 (S202). Upon receiving the print job registration request RE1 from the terminal 600, the authentication printing server 100 generates a print job J0 based on the print job registration request RE1, and performs a storage process to associate the print job J0 with the user US0 and store the print job J0 in the storage unit 104 (S204).

[0052] Furthermore, when user US0 is not logged in, the authentication printing server 100 transmits pre-login screen information to the image processing device 200, which causes the display unit 206 to display a pre-login screen D1, such as the one shown in FIG. 7 (S206). When the image processing device 200 receives the pre-login screen information from the authentication printing server 100, it performs display processing to display the pre-login screen D1 on the display unit 206 (S208). The pre-login screen D1 shown in FIG. 7 contains content that prompts user US0 to perform face authentication or card authentication. For reference, when the pre-login screen D1 is displayed on the display unit 206, if user US0 holds his or her ID card over a card reader, card authentication is performed. Here, ID is an abbreviation for identification.

[0053] When the imaging device 500 captures a contactless image of the face F0 of the user US0 who comes near the image processing device 200, it transmits a captured image IM1 of the face F0 of the user US0 to the face authentication server 400 (S210). When the face authentication server 400 receives the captured image IM1 from the imaging device 500, it transmits a face authentication in progress notification, which is a notification to start face authentication processing, to the authentication printing server 100 (S212). When the authentication printing server 100 receives the face authentication in progress notification from the face authentication server 400, it transmits face authentication in progress screen information to the image processing device 200, which causes a face authentication in progress screen D2, as exemplified in FIG. 7, to be displayed on the display unit 206 (S214). When the image processing device 200 receives the face authentication in progress screen information from the authentication printing server 100, it performs display processing to display the face authentication in progress screen D2 on the display unit 206 (S216). The face authentication in progress screen D2 shown in FIG. 7 indicates that face authentication processing is currently being performed.

[0054] The facial authentication server 400 also performs facial authentication processing based on the registered face data DA2 and the captured image IM1 (S218). As described above, the facial authentication server 400 calculates the degree of match C0 between the registered face image IM3 corresponding to the registered face data DA2 and the captured image IM1 based on the registered face data DA2 and the captured image IM1. If the degree of match C0 is equal to or greater than the threshold TH, the facial authentication server 400 generates authentication success information IN3 including information identifying the user US0 and transmits the authentication success information IN3 together with the degree of match C0 to the authentication printing server 100 (S220). When the authentication printing server 100 receives the authentication success information IN3 including information identifying the user US0 from the facial authentication server 400, the authentication printing server 100 permits the user US0 to log in and transmits function list screen information to the image processing device 200, causing the display unit 206 to display a function list screen D3 such as the one shown in FIG. 7 (S222). When the image processing device 200 receives the function list screen information from the authentication printing server 100, it performs display processing to display the function list screen D3 on the display unit 206 having the input unit 205 on the surface thereof (S224). As described above, when the user US0 is authenticated by the facial authentication process, the authentication printing server 100 causes the display unit 206 to display a function list screen D3 for accepting an instruction to execute image processing to be performed by the image processing device 200.

[0055] The function list screen D3 shown in FIG. 7 includes a print button 831 for using the print function, a copy button 832 for using the copy function, a fax button 833 for using the facsimile function, and a scan button 834 for using the read function. User US0 can use various functions of the image processing device 200 by touching the function list screen D3. When user US0 touches the print button 831, the image processing device 200 sends a print instruction for the print job J0 associated with user US0 to the authentication printing server 100 (S226). Upon receiving the print instruction from the image processing device 200, the authentication printing server 100 generates print data for printing based on the print job J0 associated with user US0 identified by the authentication success information IN3, and sends this print data to the image processing device 200 (S228). Upon receiving the print data from the authentication printing server 100, the image processing device 200 performs a print process to print in accordance with the print data (S230). As a result, printing based on the print job J0 is performed. For example, when user "US2" shown in FIG. 2 registers print job "J2" in the authentication print server 100 and points his or her face F0 toward the image capture device 500, the degree of match C0 based on the registered face data DA22 linked to the user identification information UID2 shown in FIG. 4 and the captured image IM1 becomes the largest. If this degree of match C0 is equal to or greater than a threshold TH, the face authentication server 400 generates information identifying user "US2," such as authentication success information IN3 including the user identification information UID2, and transmits the authentication success information IN3 and the like to the authentication print server 100. Upon receiving the authentication success information IN3 and the like, the authentication print server 100 receives a print instruction from the image processing device 200 in S226 and generates print data for executing printing based on the print job "J2" linked to user "US2" identified by the authentication success information IN3 and transmits the print data to the image processing device 200. Furthermore, when user US0 touches copy button 832, the image processing device 200 performs a process to copy the document. When user US0 touches fax button 833, the image processing device 200 performs facsimile communication. When user US0 touches scan button 834, the image processing device 200 performs a process to read the document. User US0 can log out of the authentication printing server 100 by touching a logout button (not shown) on the image processing device 200.

[0056] If the degree of match C0 is less than the threshold value TH, the face authentication server 400 determines that face authentication has failed and does not send authentication success information IN3. If a uniform screen informing the user of the fact that face authentication has failed is simply displayed on the display unit 206, the user US0 may not understand why face authentication has failed and may not know what to do about it. Therefore, if the user US0 is not authenticated by the facial authentication process, the authentication printing server 100 of this specific example displays a different screen depending on the level of the degree of match C0, for example, the image capture guide screen D4 or the user registration screen D5 shown in Fig. 9, on the display unit 206. This allows the user US0 to deal with the failure of facial authentication by following the image capture guide screen D4 or the user registration screen D5.

[0057] FIG. 8 illustrates a schematic example of the processing performed by the processing unit U3 when the acquisition unit U2 acquires the matching score C0. In FIG. 8, elements identical to those shown in FIG. 6 are assigned the same reference numerals. The acquisition unit U2 acquires the matching score C0 at S220 in FIG. 6 if the matching score C0 is equal to or greater than the threshold TH. If the matching score C0 is less than the threshold TH, the acquisition unit U2 acquires the matching score C0 when the facial authentication server 400 sends the matching score C0 to the authentication printing server 100 immediately after S218. Here, S240, S242, S244, and S246 correspond to the processing unit U3 and processing function FU3. FIG. 9 illustrates a schematic example of a screen displayed on the display unit 206 according to the level of the matching score C0 when the user US0 is not authenticated by the facial authentication process. 8, the degree of match C0 is divided into high, medium, and low levels, and the processing unit U3 causes the display unit 206 to display different screens depending on the level of the degree of match C0. The high level indicates that the degree of match C0 is equal to or greater than the threshold value TH, and is the level at which the face authentication server 400 authenticates the user US0. For example, if the threshold value TH is 80, the high level indicates that the degree of match C0 is equal to or greater than 80 and equal to or less than 100. Of course, the threshold value TH may be greater than 80 or less than 80.

[0058] When the user US0 is not authenticated by the facial recognition process, the level of the degree of match C0 includes a low level and a medium level that is higher than the low level and lower than the high level. Here, the low level is an example of the first level, and the medium level is an example of the second level. For example, a threshold TL is used to determine whether the degree of match is at or above the medium level. The threshold TL is smaller than the threshold TH. The medium level indicates that the degree of match C0 is equal to or higher than the threshold TL but lower than the threshold TH, at which point the user US0 may have the image capture device 500 capture the face F0 again. The low level indicates that the degree of match C0 is lower than the threshold TL, at which point the user may be registered. For example, if the threshold TH is 80 and the threshold TL is 60, the medium level indicates that the degree of match C0 is equal to or higher than 60 but lower than 80, and the low level indicates that the degree of match C0 is lower than 60. Of course, the threshold TL may be greater than or equal to 60 or less than 60.

[0059] 6, when the facial authentication server 400 calculates the degree of match C0 between the registered facial image IM3 and the captured image IM1, it sends the degree of match C0 to the authentication printing server 100 regardless of whether or not it has generated authentication success information IN3. Even if the facial authentication server 400 does not send authentication success information IN3 including information identifying user US0 to the authentication printing server 100, it may send information identifying user US0 to the authentication printing server 100 together with the degree of match C0. When the authentication printing server 100 receives the matching score C0 from the facial authentication server 400, it branches the process depending on whether the matching score C0 is high or not (S240). The facial authentication server 400 can determine that the matching score C0 is high if the matching score C0 is equal to or greater than a threshold value TH, and can determine that the matching score C0 is not high if the matching score C0 is less than the threshold value TH. Furthermore, the facial authentication server 400 may determine that the matching score C0 is high if it acquires authentication success information IN3 in S220 shown in FIG. 6, or may determine that the matching score C0 is not high if it does not acquire authentication success information IN3. If the facial authentication server 400 determines that the matching score C0 is high, it permits login by the user US0, displays the function list screen D3 shown in FIG. 7 on the display unit 206 (S222), and accepts an instruction to execute image processing to be executed by the image processing device 200.

[0060] If the face authentication server 400 determines that the degree of matching C0 is not high, it branches the process depending on whether the degree of matching C0 is medium or not (S242). The face authentication server 400 can determine that the degree of matching C0 is medium if the degree of matching C0 is equal to or greater than a threshold value TL, and can determine that the degree of matching C0 is low if the degree of matching C0 is less than the threshold value TL.

[0061] If the facial authentication server 400 determines that the matching score C0 is medium, it transmits imaging guide screen information to the image processing device 200, causing the display unit 206 to display the imaging guide screen D4 shown in FIG. 9 (S244). Upon receiving the imaging guide screen information from the authentication printing server 100, the image processing device 200 performs display processing to display the imaging guide screen D4 on the display unit 206, which has the input unit 205 on its surface. If the matching score C0 is medium, the display on the display unit 206 switches from the facial authentication in progress screen D2 to the imaging guide screen D4. The imaging guide screen D4 shown in FIG. 9 includes a display area that reads, "Authentication failed. Please hold your face over the screen again." and an OK button 841. Therefore, the imaging guide screen D4 indicates that the user US0 was not authenticated by the facial authentication process and prompts the user to take an image to perform the facial authentication process again. Upon receiving an operation of the OK button 841 from the user US0, the image processing device 200 transmits a confirmation notification to the authentication printing server 100. When the authentication printing server 100 receives the confirmation notification from the image processing device 200, it ends the process shown in FIG.

[0062] By looking at the imaging guide screen D4, the user US0 can understand that they should perform the action of capturing an image of their face F0 again. When the user US0, looking at the imaging guide screen D4, points their face F0 toward the imaging device 500, the imaging device 500 captures the face F0 of the user US0 in S210 shown in Fig. 6 and transmits the captured image IM1 to the face authentication server 400. Thereafter, the processes of S212 to S218 are performed, and when the degree of match C0 reaches a high level, the function list screen D3 shown in Fig. 7 is displayed on the display unit 206, and an instruction to execute image processing is accepted. The processing system SY1 of this specific example is convenient because the imaging guide screen D4 is displayed on the display unit 206 when the degree of coincidence C0 is lower than the high level but higher than the low level.

[0063] If the facial authentication server 400 determines that the degree of match C0 is low, it transmits user registration screen information to the image processing device 200, which causes the display unit 206 to display the user registration screen D5 shown in FIG. 9 (S246). Upon receiving the user registration screen information from the authentication printing server 100, the image processing device 200 performs display processing to display the user registration screen D5 on the display unit 206, which has the input unit 205 on its surface. If the degree of match C0 is low, the display on the display unit 206 switches from the facial authentication in progress screen D2 to the user registration screen D5. The user registration screen D5 shown in FIG. 9 includes a display area that reads, "User registration not completed. Please register the user." and an OK button 851. Therefore, the user registration screen D5 indicates that the user US0 was not authenticated by the facial authentication process and prompts the user to generate registration face data DA2 (see FIGS. 4 and 5) based on the original facial image IM2 of the user US0, which is linked to the user identification information UID. When the image processing device 200 accepts operation of the OK button 851 from the user US0, it sends a confirmation notice to the authentication printing server 100. When the authentication printing server 100 receives the confirmation notice from the image processing device 200, it ends the process shown in FIG.

[0064] By viewing the user registration screen D5, user US0 can understand that user registration is required. After viewing the user registration screen D5, user US0 can register by starting the user registration process shown in FIG. 4. According to the user registration process, the face authentication server 400 acquires user identification information UID and the original face image IM2 shown in FIG. 5, generates registration face data DA2 based on the original face image IM2, associates the registration face data DA2 with the user identification information UID, and stores the registration face data DA2 in the face database DB2. When user US0 points his / her face F0 toward the image capture device 500 again, in S210 shown in FIG. 6, the image capture device 500 captures the face F0 of user US0 and transmits the captured image IM1 to the face authentication server 400. After that, the processes of S212 to S218 are performed. When the degree of match C0 reaches a high level, the function list screen D3 shown in FIG. 7 is displayed on the display unit 206, and an instruction to execute image processing is accepted. The processing system SY1 of this example is convenient because the user registration screen D5 is displayed on the display unit 206 when the degree of coincidence C0 is a low level lower than the medium level.

[0065] As described above, when the user US0 is not authenticated by the facial recognition process, instead of a uniform screen, a different screen is displayed on the display unit 206 of the image processing device 200 depending on the level of the degree of match C0 between the registered facial image IM3 and the captured image IM1. The user US0 can understand the level of the degree of match C0 between the registered facial image IM3 and the captured image IM1 by looking at the screen of the display unit 206, for example, the imaging guide screen D4 or the user registration screen D5. Therefore, this specific example can improve the convenience of the user US0 who uses the image processing device 200.

[0066] (5) Variation: The present invention can be modified in various ways. For example, even if the degree of match C0 is high, the facial authentication server 400 may not send the authentication success information IN3 to the authentication printing server 100, but may instead send the degree of match C0 together with information identifying user US0 to the authentication printing server 100. If the degree of match C0 is equal to or greater than the threshold value TH, this means that user US0 has been authenticated by facial authentication processing. Therefore, if the degree of match C0 is equal to or greater than the threshold value TH, the authentication printing server 100 may permit user US0 to log in and display the function list screen D3 on the display unit 206. The authentication printing server 100 may also include a face authentication unit U1.

[0067] The above-described processes can be changed as appropriate, such as by changing the order of the processes. For example, in the process shown in FIG. 4, the face authentication server 400 may perform the user identification information acquisition process of S102 after the process of S104 or S106. In the process shown in FIG. 8, the processing unit U3 may perform the determination process of S242 before the determination process of S240. Furthermore, the above-described processes are not limited to being performed by a CPU, but may also be performed by an electronic component such as an ASIC. Here, ASIC is an abbreviation for Application Specific Integrated Circuit. Furthermore, the above-described processes may be distributed among multiple CPUs, or may be performed by a CPU and an electronic component such as an ASIC operating in cooperation with each other.

[0068] As shown in Fig. 10, if the image capturing device 500 repeatedly captures the face F0 of user US0 and the matching score C0 is repeatedly found to be medium, the authentication printing server 100 may display a user registration screen D5 on the display unit 206. Fig. 10 schematically illustrates another process performed by the processing unit U3 when the acquiring unit U2 acquires the matching score C0. In Fig. 10, elements that are the same as those shown in Fig. 8 are assigned the same reference numerals. Compared to the process shown in Fig. 8, the process shown in Fig. 10 adds S250.

[0069] If the matching score C0 received from the facial authentication server 400 is medium, the authentication printing server 100 proceeds to S250 via the processes of S240 and S242. In S250, the authentication printing server 100 branches the process depending on whether the matching score C0 obtained N consecutive times for the user being processed is medium. Here, N is an integer greater than or equal to 2. To determine the number of consecutive medium-level matching scores for the user being processed, the point in time when a medium-level matching score C0 is obtained for the user being processed after a matching score C0 has been obtained for a user other than the user being processed is used as the starting point for counting the number of consecutive medium-level matching scores. If the degree of match C0 acquired for the user being processed N consecutive times is at a medium level, the authentication printing server 100 causes the display unit 206 to display the user registration screen D5 shown in Fig. 9 (S246). On the other hand, if the degree of match C0 acquired for the user being processed is at a medium level less than N consecutive times, the authentication printing server 100 causes the display unit 206 to display the imaging guide screen D4 shown in Fig. 9 (S244).

[0070] From the above, if the number of consecutive times that the degree of matching C0 obtained for user US0 is at a medium level is less than N, user US0 can understand by looking at the imaging guide screen D4 that he or she should take another action to capture the face F0. Even if the level of the degree of match C0 is medium, which is higher than low, if the degree of match C0 calculated in the face recognition process is repeatedly medium, the possibility of successful face recognition is low. When the degree of match C0 obtained for user US0 is medium N times in a row, user US0 can see by looking at the user registration screen D5 that he or she should register as a user.

[0071] As described above, the example shown in FIG. 10 can further improve the convenience of the user US0 who uses the image processing device 200.

[0072] As illustrated in FIG. 11, the processing system SY1 may not include the authentication printing server 100, and the acquisition unit U2 and processing unit U3 may be included in the image processing device 200. FIG. 11 schematically illustrates another configuration of a system including the processing system SY1. The system includes an image processing device 200, a facial authentication server 400, an imaging device 500, and a terminal 600. The image processing device 200 includes the acquisition unit U2 and processing unit U3, etc. Therefore, the processing system SY1 is included in the image processing device 200. The acquisition unit U2, which includes a network I / F 211, acquires a print job registration request RE1, a matching score C0, authentication success information IN3, etc. When the user US0 is authenticated by facial authentication processing, the processing unit U3 allows the user US0 to log in and displays the function list screen D3 shown in FIG. 7 on the display unit 206 of the image processing device 200. Furthermore, if the user US0 is not authenticated by the face authentication process, the processing unit U3 causes the display unit 206 to display the imaging guide screen D4 or the user registration screen D5 shown in FIG. 9 according to the level of the degree of coincidence C0.

[0073] 11 can also improve convenience for the user US0 who uses the image processing device 200. The image processing device 200 may include a face authentication unit U1.

[0074] (6) Conclusion: As described above, the present invention provides, in various aspects, techniques for improving the convenience of users US0 who use image processing device 200. Of course, even a technique consisting only of the constituent elements of the independent claims can achieve the basic actions and effects described above. Furthermore, it is possible to implement configurations in which the components disclosed in the above examples are substituted with each other or the combination is changed, or configurations in which the components disclosed in the publicly known techniques and the above examples are substituted with each other or the combination is changed, etc. The present invention also includes these configurations. [Explanation of symbols]

[0075] 100...authentication printing server, 104...memory unit, 107...network I / F, 200...image processing device, 205...input unit, 206...display unit, 208...printing unit, 400...face authentication server, 500...imaging device, 600...terminal, C0...match degree, D0...screen information, D1...pre-login screen, D2...face authentication in progress screen, D3...function list screen, D4...imaging guide screen, D5...user registration screen, DA1...captured face data, DA2...registered face data, DB2...face database, F0...face, FU2...acquisition function, FU3...processing function, IM1...captured image, IM2...original face image, IM3...registered face image, IN3...authentication success information, NE1...network, PR1...control program, SY1...processing system, U1...face authentication unit, U2...acquisition unit, U3...processing unit, UID...user identification information, US0...user.

Claims

1. A processing system including an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; The processing unit, when the degree of match acquired by the acquisition unit is the second level, causes the display unit to display an imaging guide screen that prompts the user to take another image to perform the face recognition process.

2. A processing system including an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; N is an integer equal to or greater than 2, The processing unit When the degree of match acquired by the acquisition unit for the user N consecutive times is at the second level, displaying on the display unit a user registration screen that prompts the user to generate the face data to be linked to user identification information that identifies the user; If the degree of match acquired by the acquisition unit is at the second level and the user registration screen is not displayed, the processing system causes the display unit to display an imaging guidance screen that prompts the user to take another image to perform the face recognition process.

3. 3. The processing system according to claim 1, wherein the processing unit causes the display unit to display a screen via a network.

4. The processing system according to claim 1 , wherein the acquisition unit and the processing unit are included in the image processing device.

5. An information processing device connected to an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; When the degree of match acquired by the acquisition unit is the second level, the processing unit causes the display unit to display an imaging guide screen that prompts the user to capture an image to perform the face recognition processing again.

6. An information processing device connected to an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; N is an integer equal to or greater than 2, The processing unit When the degree of match acquired by the acquisition unit for the user N consecutive times is at the second level, displaying on the display unit a user registration screen that prompts the user to generate the face data to be linked to user identification information that identifies the user; When the degree of match acquired by the acquisition unit is at the second level and the user registration screen is not displayed, an image capture guide screen is displayed on the display unit to prompt the user to capture an image again to perform the face authentication processing.

7. A control program for controlling an image processing device having a display unit, an acquisition function for acquiring the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing function of displaying, on the display unit, an execution instruction screen for receiving an execution instruction for image processing to be executed by the image processing device when the user is authenticated by the face authentication processing, the processing function displays different screens on the display unit depending on the level of the degree of match when the user is not authenticated by the face authentication processing; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; The processing function is a control program that, when the degree of match acquired by the acquisition function is the second level, causes the display unit to display an imaging guidance screen that prompts the user to take another image to perform the facial recognition processing.

8. A control program for controlling an image processing device having a display unit, an acquisition function for acquiring the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing function of displaying, on the display unit, an execution instruction screen for receiving an execution instruction for image processing to be executed by the image processing device when the user is authenticated by the face authentication processing, the processing function displays different screens on the display unit depending on the level of the degree of match when the user is not authenticated by the face authentication processing; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; N is an integer equal to or greater than 2, The processing function is When the degree of match acquired by the acquisition function for the user N consecutive times is the second level, displaying a user registration screen on the display unit to prompt generation of the face data to be linked to user identification information that identifies the user, A control program that, when the degree of match acquired by the acquisition function is the second level and the user registration screen is not displayed, displays an imaging guidance screen on the display unit to prompt the user to take another image to perform the face recognition processing.

9. A display unit; an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an instruction to execute image processing, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; When the degree of match acquired by the acquisition unit is the second level, the processing unit causes the display unit to display an imaging guide screen that prompts the user to capture an image to perform the face recognition process again.

10. A display unit; an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an instruction to execute image processing, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; N is an integer equal to or greater than 2, The processing unit When the degree of match acquired by the acquisition unit for the user N consecutive times is at the second level, displaying on the display unit a user registration screen that prompts the user to generate the face data to be linked to user identification information that identifies the user; When the degree of match acquired by the acquisition unit is at the second level and the user registration screen is not displayed, an image acquisition guide screen is displayed on the display unit to prompt the user to take another image to perform the face recognition processing.

11. A processing system including an image processing device having a display unit, an acquisition unit that acquires the degree of match from a face authentication unit that performs face authentication processing based on the degree of match between a face image corresponding to registered face data and a captured image of the user's face; a processing unit that, when the user is authenticated by the face authentication processing, causes the display unit to display an execution instruction screen that accepts an execution instruction for image processing to be executed by the image processing device, When the user is not authenticated by the face authentication process, the processing unit displays a different screen on the display unit according to the level of the degree of match; the level of the degree of match when the user is not authenticated by the face authentication processing includes a first level and a second level higher than the first level; The processing unit, when the degree of match acquired by the acquisition unit is the first level, causes the display unit to display a user registration screen that prompts the user to generate face data that will be linked to user identification information that identifies the user.

Images