Information processing apparatus, storage medium, information processing method, and computer program product

Abstract

Provided is an information processing device, storage medium, information processing method, and computer program product, which can operate execution processing in accordance with an authentication result and a recognition result of a user's voice even when the user is not authenticated or the user's voice is not recognized. The information processing device is characterized by including a processor that performs the following steps: acquires a user's voice; authenticates the user using the voice and recognizes the voice; and displays, on a display unit, an operation screen that is different in accordance with an authentication result of the user and a recognition result of the voice and is used in operation of execution processing.

Description

TECHNICAL FIELD

[0001] The present application relates to an information processing apparatus, a storage medium, an information processing method, and a computer program product. BACKGROUND

[0002] A technique of operating an apparatus using a voice is known. For example, Patent Literature 1 describes a technique of performing voiceprint authentication by acquiring a user's voice, and executing processing in accordance with text data obtained by performing voice recognition on the voice. Patent Literature 2 describes a technique of generating a command by receiving a voice and performing voice recognition when a person approaches, and supplying power to a functional section required when the command is executed. Patent Literature 3 describes a technique of extracting a keyword by recognizing a user's voice, and interacting with the user in a dialogue form when the keyword coincides with a keyword registered in a voiceprint pattern file, and performing an action of image formation in accordance with the user's voice instruction.

[0003] Patent Literature 1: Japanese Patent Application Laid-Open No. 2007-188001

[0004] Patent Literature 2: Japanese Patent Application Laid-Open No. 2017-028355

[0005] Patent Literature 3: Japanese Patent Application Laid-Open No. 2007-193138 SUMMARY

[0006] As the technique described in Patent Literature 1 above, when processing is executed only in a case where both authentication of a user and voice recognition are successful, an operation for executing processing in a case where the user is not authenticated or the user's voice is not recognized is not accepted, and the processing is not executed.

[0007] An object of the present application is to enable operation of execution of processing in accordance with a result of authentication of a user and a result of recognition of a voice even in a case where the user is not authenticated or the user's voice is not recognized.

[0008] The application described in Aspect 1 is an information processing apparatus characterized by comprising a processor that performs the steps of: acquiring a user's voice; authenticating the user using the voice, and recognizing the voice; and displaying, on a display section, an operation screen that differs in accordance with a result of authentication of the user and a result of recognition of the voice, and is used in operation of execution of processing.

[0009] The application described in Aspect 2 is the information processing apparatus described in Aspect 1, characterized in that, when the user is authenticated, the operation screen differs in accordance with the result of recognition of the voice, and the operation screen includes a set value of the processing that is set in advance for the authenticated user.

[0010] The invention described in Scheme 3, in the information processing device described in Scheme 2, is characterized in that, when the setting value is not preset, the operation screen includes the setting value previously used by the user in the processing.

[0011] The invention described in Scheme 4, in the information processing device described in Scheme 1, is characterized in that, when the user is authenticated, the processor does not display the operation screen, and performs the processing even if no operation is performed.

[0012] The invention described in Scheme 5, in the information processing apparatus described in Scheme 4, is characterized in that, when the user does not perform the work required to execute the processing, the processor displays information prompting the user to perform the work on the display unit.

[0013] The invention described in Scheme 6, in the information processing apparatus described in Scheme 1, is characterized in that the processor performs user authentication and voice recognition based on the detection of the user located on the front of the information processing apparatus.

[0014] The invention described in Scheme 7, in the information processing apparatus described in Scheme 1, is characterized in that, when the user is identified without authenticating the user using the voice, the processor uses an image of the user captured by the camera unit to authenticate the user, and when authenticating the user using the image, the operation screen is displayed.

[0015] The invention described in Scheme 8, in the information processing apparatus described in Scheme 1, is characterized in that, when the user is not authenticated using the voice, the processor displays the authentication screen used in the authentication of the user through other authentication methods on the display unit.

[0016] The invention described in Scheme 9, in the information processing apparatus described in Scheme 8, is characterized in that, when the user is authenticated by the other authentication method after the voice has been recognized, the processor displays the operation screen on the display unit.

[0017] The invention described in Scheme 10, in the information processing apparatus described in Scheme 1, is characterized in that, when the voice is recognized without authenticating the user, the processor displays a second operation screen with a restricted setting value on the display unit, compared to the first operation screen displayed when the user is authenticated.

[0018] The invention described in Scheme 11, in the information processing apparatus described in Scheme 1, is characterized in that, when the voice is recognized without authenticating the user, the processor restricts its functions compared to when the user is authenticated.

[0019] The invention described in Scheme 12 is a storage medium storing a program for causing a computer to perform the following steps: acquiring a user's voice; authenticating the user using the voice and recognizing the voice; and displaying an operation screen, which varies depending on the user's authentication result and the voice recognition result, on a display unit during the processing.

[0020] The invention described in Scheme 13 is an information processing method, characterized by comprising the following steps: acquiring the user's voice; authenticating the user using the voice and recognizing the voice; and displaying an operation screen, which varies depending on the user's authentication result and the voice recognition result, and is used in the processing operation on a display unit.

[0021] Invention Effects

[0022] According to the first aspect of the present invention, even if the user is not authenticated or the user's voice is not recognized, the execution process can be operated based on the user's authentication result and the voice recognition result.

[0023] According to the second aspect of the present invention, the setting value can be used easily compared to the case where the setting value for the user's pre-set processing is not included in the operation screen.

[0024] According to the third aspect of the present invention, the setting value can be easily used compared to the case where the setting value used by the user in the previous process is not included in the operation screen.

[0025] According to the fourth aspect of the present invention, the user performs the processing without using the operation screen.

[0026] According to the fifth aspect of the present invention, the work required for the user to perform the processing is known.

[0027] According to the sixth aspect of the present invention, it is possible to prevent user authentication and voice recognition from being performed by the voice of a person other than the user of the information processing device.

[0028] According to the seventh aspect of the present invention, the information processing device can be used even when the user is not authenticated when using voice.

[0029] According to the eighth aspect of the present invention, the information processing device can be used even when the user is not authenticated when using voice.

[0030] According to the ninth aspect of the present invention, even when the user is not authenticated using voice, the processing can be performed based on the user's authentication result and the voice recognition result.

[0031] According to the tenth aspect of the present invention, processing can be performed according to the restricted settings even without authenticating the user.

[0032] According to the eleventh aspect of the present invention, it is possible to perform processes with limited functionality even without authenticating the user.

[0033] According to the 12th aspect of the present invention, even if the user is not authenticated or the user's voice is not recognized, the execution process can be operated based on the user's authentication result and the voice recognition result.

[0034] According to the 13th aspect of the present invention, even if the user is not authenticated or the user's voice is not recognized, the execution process can be operated based on the user's authentication result and the voice recognition result. Attached Figure Description

[0035] The embodiments of the present invention will be described in detail with reference to the following figures.

[0036] Figure 1 A diagram illustrating an example of the structure of the image processing system 100 according to an embodiment;

[0037] Figure 2 A diagram illustrating an example of the structure of the image processing apparatus 120;

[0038] Figure 3 A diagram illustrating an example of speech database 211;

[0039] Figure 4 A flowchart illustrating an example of the operation of the image processing apparatus 120;

[0040] Figure 5 This is an example of a login screen (140).

[0041] Figure 6 This is an example of a screen setting of 150.

[0042] Figure 7 This is an example diagram showing a menu screen at angle 160.

[0043] Figure 8 This is an example of a screenshot showing the authentication result (screen number 170).

[0044] Figure 9A sequence diagram illustrating an example of the setting process for a restricted mode;

[0045] Figure 10 An example diagram of screen 175 showing the fax sending address;

[0046] Figure 11 An example of a notification screen at 180 degrees;

[0047] Figure 12 An example of a setting screen 185 showing the limitation mode when object processing is a copy processing;

[0048] Figure 13 This is an example of a setting screen 190 showing the limitation mode when the object is processed as a fax.

[0049] Symbol Explanation

[0050] 100 - Image processing system; 110 - Server device; 120 - Image processing device; 121 - Processor; 122 - Memory; 123 - Communication unit; 124 - Operation unit; 125 - Display unit; 126 - Image reading unit; 127 - Image forming unit; 128 - Voice acquisition unit; 129 - Camera unit. Detailed Implementation

[0051] 1. Composition

[0052] Figure 1 This diagram illustrates an example of the structure of the image processing system 100 according to an embodiment. The image processing system 100 includes a server device 110 and an image processing device 120. The server device 110 and the image processing device 120 are connected via a communication line 130 such as the Internet. The image processing device 120 authenticates the user using the user's voice, recognizes the user's voice, and receives operations (hereinafter referred to as "voice operations") based on the recognition results. When user authentication fails but voice recognition succeeds, the server device 110 determines whether to allow the image processing device 120 to perform processing based on the image that is the object of processing by the image processing device 120. The server device 110 has the function of recognizing characters contained in an image. This function is implemented, for example, through OCR (Optical Character Recognition).

[0053] Figure 2This figure illustrates an example of the structure of an image processing apparatus 120. The image processing apparatus 120 has multiple functions for processing images, including copying, printing, scanning, and faxing. The image processing apparatus 120 performs these functions based on user input. These inputs include voice input and screen input. The image processing apparatus 120 includes a processor 121, a memory 122, a communication unit 123, an operation unit 124, a display unit 125, an image reading unit 126, an image forming unit 127, a voice acquisition unit 128, and a camera unit 129. These components are connected via a bus.

[0054] The processor 121 controls various parts of the image processing apparatus 120 to perform processing that realizes the functions of the image processing apparatus 120 by executing a program. The processor 121 uses, for example, a CPU (Central Processing Unit). The memory 122 stores programs for executing the processing of the image processing apparatus 120 and a voice database 211 storing pre-registered user voices. The memory 122 uses, for example, ROM (Read Only Memory) and RAM (Random Access Memory). In addition to ROM and RAM, the memory 122 may also use, for example, a hard disk drive or an SSD (Solid State Drive). The communication unit 123 is connected to the communication line 130. The communication unit 123 communicates with the server device 110 via the communication line 130. The operation unit 124 is used for operation of the image processing apparatus 120 by the user. The operation unit 124 uses, for example, a touch panel and buttons. The display unit 125 displays various screens related to the processing of the image processing apparatus 120. The display unit 125 uses, for example, a liquid crystal display (LCD). Image reading unit 126 reads an image and converts it into image data. Image reading unit 126 may use, for example, an image scanner. Image forming unit 127 forms an image corresponding to the image data on a medium such as paper. Image forming unit 127 may use, for example, a printer. Voice acquisition unit 128 acquires the user's voice. Voice acquisition unit 128 may use, for example, a microphone. Camera unit 129 captures an image of the user. Camera unit 129 may use, for example, a camera.

[0055] Figure 3This diagram illustrates an example of a voice database 211. The voice database 211 contains a user ID, a process ID, and a voice message. The user ID uniquely identifies the user. The process ID uniquely identifies the process. This process ID also serves as a command for selecting or executing a process. The voice message is the voice of the operation word for each process issued by the user. This voice message is pre-registered by the user. Specifically, the user pre-determines the operation word for each process and registers the voice message of that operation word in the voice database 211. For example, when user A determines the operation word for the copying process as "copy," user A issues the voice message "copy." This voice message is acquired by the voice acquisition unit 128 and converted into a digital signal, such as... Figure 3 As shown, a group is established and stored that associates the user ID of user A with the processing ID of the copying process. Furthermore, when user A selects "fax" as the operation word for the fax processing, user A utters the word "fax". This voice is acquired by the voice acquisition unit 128 and converted into a digital signal, as shown... Figure 3 As shown, the user ID of user A is associated with the processing ID of fax processing and stored.

[0056] Furthermore, the speech database 211 is not limited to the user's own speech; it can contain features of the user's speech or a machine learning model obtained by performing machine learning on the user's speech. For example, the machine learning model could be generated by having the user pronounce the words of an operand multiple times and then performing machine learning on those pronunciations. When input speech is received, the machine learning model determines whether it corresponds to the user's speech.

[0057] 2. Actions

[0058] In the following description, when processor 121 is referred to as the subject of processing, it means that processor 121 performs calculations or controls the actions of other hardware components in cooperation with a program stored in memory 122 and the processor 121 executing the program.

[0059] Figure 4 This is a flowchart illustrating an example of the operation of the image processing device 120. This operation begins, for example, when a user uses the image processing device 120. In step S11, the processor 121 displays the login screen 140 on the display unit 125. Figure 5 This is an example of a login screen 140. For example, when copying an original image, if the operation word for the copying process is "copy", the user pronounces the word "copy".

[0060] In step S12, the processor 121 acquires the user's voice through the voice acquisition unit 128. For example, when the user utters the voice "copy", the voice acquisition unit 128 acquires that voice.

[0061] In step S13, processor 121 uses the speech acquired in step S12 to perform user authentication processing and speech recognition processing. These processes are performed in parallel and simultaneously. That is, speech recognition processing is performed regardless of the result of user authentication processing. Furthermore, the term "simultaneously" here does not necessarily mean exactly the same time; there may be slight deviations.

[0062] Voice authentication is used to authenticate users during the user authentication process. For example, the voiceprint of the voice obtained in step S12 is compared with the voiceprints of voices stored in the voice database 211 in memory 122. Here, when the voiceprint of the voice obtained in step S12 matches the voiceprint of the voice associated with user A's user ID in the voice database 211, user A is authenticated and the user authentication is successful. On the other hand, when there is no voice in the voice database 211 that matches the voice and voiceprint obtained in step S12, the user is not authenticated and the user authentication fails. For example, for users who have not pre-registered, there is no voice registered in the voice database 211, so the user authentication fails. Furthermore, when the voice database 211 contains machine learning models of each user's voice, the voice obtained in step S12 can be input into these machine learning models to determine whether it is the voice of any user.

[0063] In speech recognition processing, the user's speech is recognized. The term "speech recognition" here is not limited to recognizing the operation word itself represented by the speech, but also includes recognizing the meaning of the object being processed represented by the operation word. For example, the speech obtained in step S12 is compared with the speech contained in the speech database 211 stored in memory 122. Here, when the speech obtained in step S12 matches the speech associated with the user A's user ID and the copy processing ID in the speech database 211, the operation word for user A's copy processing is recognized. Furthermore, when the speech database 211 contains machine learning models of each user's speech, the speech obtained in step S12 can be input into these machine learning models to determine whether it is an operation word for arbitrary processing. On the other hand, for example, when the user speaks in a language other than the operation word, the operation word is not recognized, and speech recognition fails.

[0064] In step S14, the processor 121 determines the results of the user authentication process and the speech recognition process performed in step S13. For example, if both user authentication and speech recognition fail (the determination in step S14 is "user authentication failed, speech recognition failed"), the process returns to the process in step S11. At this time, a login failure message can be displayed on the display unit 125. On the other hand, if both user authentication and speech recognition succeed (the determination in step S14 is "user authentication successful, speech recognition successful"), the process proceeds to step S15.

[0065] In step S15, the processor 121 displays an object processing setting screen 150 on the display unit 125, which displays the operation words recognized by the speech recognition process. This setting screen 150 is used to perform object processing operations. The object processing operations include setting up the object processing. Furthermore, the setting screen 150 may contain user-specific information identified through user authentication. The setting screen 150 is an example of an "operation screen" or "first operation screen" according to the present invention.

[0066] Figure 6 This diagram illustrates an example of setting screen 150. Here, user A is identified through user authentication processing, and the operation word for copying by user A is recognized through voice recognition processing. In this case, as... Figure 6 As shown, a settings screen 150 is displayed for use in setting up copy processing. The user uses the settings screen 150 to perform copy processing settings. Furthermore, the settings screen 150 can be a screen exclusively for an authenticated user A. For example, when user A presets copy processing settings, such as... Figure 6 As shown, the setting screen 150 may include a setting button 151 representing the setting value. This setting value may be a set of multiple setting values. The user can use the preset copy processing settings by pressing the setting button 151. On the other hand, if the user A has not preset the copy processing settings, the setting screen 150 may include a setting button representing the setting value used by user A in the previous copy processing. In this case, the memory 122 stores the usage history of the setting values ​​for each processing for each user, and determines the user's last used setting value based on this usage history.

[0067] Furthermore, when the operation word for fax processing by user A is recognized through voice recognition processing, a settings screen used in the fax processing settings operation is displayed. Similarly, when the operation word for scan processing by user A is recognized through voice recognition processing, a settings screen used in the scan processing settings operation is displayed. Thus, in step S15, different settings screens are displayed based on the result of voice recognition.

[0068] When the result determined in step S14 is that user authentication is successful but speech recognition fails (the determination in step S14 is "user authentication successful, speech recognition failed"), the process proceeds to step S16. In step S16, the processor 121 displays a user-specific menu screen 160, which has been authenticated through the user authentication process, on the display unit 125. The menu screen 160 is used to perform operations of the image processing device 120. This operation includes selecting the processing of the image processing device 120. Furthermore, the menu screen 160 contains user-specific information that has been identified through the user authentication process. The menu screen 160 is an example of an "operation screen" according to the present invention.

[0069] Figure 7 This is an example diagram representing menu screen 160. Here, user A is identified through user authentication processing. (Example...) Figure 7 As shown, menu screen 160 includes selection buttons 161-163 used for selecting operations such as copying, scanning, and faxing. Selection buttons 161-163 accept operations to select the processing of the image processing device 120. User A can select any of these processing options by selecting any of the selection buttons 161-163. If any processing of the image processing device 120 is selected, a setting screen for that processing is displayed. For example, if selection button 161 is pressed, copying is selected, and therefore the following screen is displayed: Figure 6 The copy processing settings screen 150 is shown.

[0070] Furthermore, when user A presets settings for copying or scanning, menu screen 160 includes selection buttons 164 and 165 for selecting copying or scanning based on the preset settings. These settings can be a set of multiple settings. User A can select copying or scanning based on the preset settings by pressing selection buttons 164 or 165.

[0071] Moreover, such as Figure 7 As shown, the menu screen 160 may include guidance information 166 for voice operation. The guidance information 166 may contain operation words for specific processes. This specific process could be, for example, a process where the usage frequency of user A is above a threshold. In this case, the usage frequency of each process is calculated for each user and stored in the memory 122. For example, when the usage frequency of the copying process based on user A is above a threshold, the guidance information 166 includes an operation word indicating that... Figure 3The speech database 211 shown contains the speech word "copy" associated with the user ID of user A and the processing ID of the copying process.

[0072] If the result determined in step S14 is that user authentication failed but speech recognition succeeded (the determination in step S14 is "user authentication failed, speech recognition succeeded"), the process proceeds to step S17. In step S17, the processor 121 determines whether to re-authenticate the user. At this time, the processor 121 displays the authentication result screen 170, indicating the authentication result, on the display unit 125. Figure 8 This is an example of a screenshot showing the authentication result (screen number 170). Again, user authentication failed, but the word "copy" (or "copy") was recognized through voice recognition processing. (Example follows) Figure 8 As shown, the authentication result screen 170 includes an authentication result indicating a failure in user authentication and a speech recognition result of "copy". Furthermore, the authentication result screen 170 includes a selection button 171 for selecting to re-authenticate the user and a selection button 172 for selecting to perform processing in restricted mode. This restricted mode refers to a mode of operation that restricts functionality compared to a successful user authentication. In restricted mode, processing deemed safe from an information security perspective is allowed. That is, in restricted mode, processing deemed problematic from an information security perspective is prohibited. Thus, even in the case of a failed user authentication, processing in restricted mode can be performed because the user cannot be identified, but speech recognition was successful, and the user's desired processing was recognized; therefore, processing is considered permissible within a range deemed safe from an information security perspective.

[0073] When re-authenticating, the user presses the selection button 171. If this action is performed, it is determined that user authentication is to be re-authenticated (the determination in step S17 is "Yes"), and the process returns to the previous step S11. On the other hand, when the user wishes to perform processing in restricted mode without re-authenticating, they press the selection button 172. If this action is performed, it is determined that user authentication will not be re-authenticated (the determination in step S17 is "No"), and the process proceeds to the restricted mode setting process in step S18.

[0074] Figure 9This is a sequence diagram illustrating an example of the setting process for a restricted mode. Furthermore, the server device 110, like the image processing device 120, includes a processor, memory, and a communication unit. In the following description, when the server device 110 is described as the main processing unit, this means that processing is performed by the processor cooperating with a program stored in memory and the processor executing that program, allowing the processor to perform calculations or control the operation of other hardware components.

[0075] In step S21, the processor 121 of the image processing apparatus 120 sends a request to the server apparatus 110 asking whether object processing can be performed. This request includes image data representing the image used in the object processing. For example, when the operation word for copying is recognized through voice recognition processing, the object processing becomes copying; therefore, the request includes image data representing the original image read by the image reading unit 126. On the other hand, when the operation word for fax processing is recognized in step S13 above, the object processing becomes fax processing. In this case, the processor 121 first displays the fax sending address setting screen 175 on the display unit 125. Figure 10 This diagram illustrates an example of a fax sending address setting screen 175. Setting screen 175 accepts the operation of specifying a fax sending address. However, setting screen 175 does not accept the operation of specifying a sending address only for a pre-registered sending address. For example, as... Figure 10 As shown, the setup screen 175 includes selection buttons 176 and 177 for accepting the selection of pre-registered sending addresses, namely Company X and Company Y. On the other hand, the setup screen 175 does not include an operation image for accepting the selection of a new sending address, and such operation is not accepted. The user uses the setup screen 175 to select a sending address. For example, if the user presses selection button 176, the fax sending address is selected as "Company X". In this case, the query request includes image data representing the original image read by the image reading unit 126 and sending address information representing the sending address "Company X". Furthermore, this query request is not always required. For example, when the object processing is scanning, the query request may not be performed.

[0076] In step S22, the server device 110 determines whether object processing is permitted based on the image data received from the image processing device 120. For example, the server device 110 recognizes characters contained in the image data using OCR. For example, when the image data contains specific characters, it is determined that object processing is prohibited. These specific characters may be, for example, "company secret" or "copying prohibited," and can be preset for each processing step. Furthermore, for example, when sending address information is received simultaneously with the image data, it is determined that object processing is prohibited if the character column representing the sending address contained in the image data does not represent the sending address represented by the sending address information. On the other hand, in other cases, it is determined that object processing is permitted. In step S23, the server device 110 sends a response to the image processing device 120 indicating the determination result of step S22.

[0077] In step S24, the processor 121 of the image processing apparatus 120 determines whether object processing is permitted based on the response received from the server device 110. For example, if the response from the server device 110 indicates a determination that object processing is prohibited, then object processing is prohibited (the determination in step S24 is "No"). In this case, in step S25, the processor 121 displays a notification screen 180 indicating that object processing is not permitted on the display unit 125. Figure 11 This diagram illustrates an example of a notification screen 180 displayed when an object is being processed as a copy. Figure 11 As shown, notification screen 180 includes a message indicating that copying will not be performed.

[0078] On the other hand, in step S24 above, when the response from server device 110 indicates that the object processing is permitted, it is determined that the object processing is permitted (the determination in step S24 is "yes"). In this case, in step S26, processor 121 displays the object processing setting screen in restricted mode on display unit 125. Similar to the setting screen 150 displayed when user authentication is successful, this restricted mode setting screen is used to perform object processing operations on operation words recognized by voice recognition processing. This operation includes setting the object processing. However, in the restricted mode setting screen, the settable values ​​are limited compared to the setting screen 150 displayed when user authentication is successful. In the restricted mode setting screen, only setting values ​​that are considered acceptable from an information security point of view can be set. In other words, in the restricted mode setting screen, setting values ​​that are considered problematic from an information security point of view cannot be set. The restricted mode setting screen is an example of the "second operation screen" according to the present invention. The setting values ​​that can be set in restricted mode are predetermined for each process.

[0079] Regarding copying and printing, for example, color is limited to black and white, and the number of copies is limited to a maximum. Furthermore, regarding printing, only USB (Universal Serial Bus) printing is allowed, and network printing is restricted. USB printing refers to the function of printing images corresponding to image data stored in a USB memory. Network printing refers to the function of printing images corresponding to image data transmitted via communication lines such as the Internet. Since network printing can be used by an unspecified number of people, USB printing is limited to users located near the image processing device 120, thus the information security risk of USB printing is considered low. Regarding faxing, for example, the sending address is limited to a pre-registered sending address, the print source information cannot be disabled, and redialing is not allowed. Redialing refers to the function of automatically redialing when the initial fax transmission fails. Regarding scanning, for example, the storage address is limited to a guest storage address, and the image attributes are limited to read-only attributes. When the image attributes are read-only, the image cannot be edited.

[0080] Figure 12 This diagram illustrates an example of a setting screen 185 displaying a restricted mode when the object is processed as a copy. This setting screen 185 is used for setting up the copy process. However, compared to... Figure 6 Compared to the setup screen 150, the setup screen 185 is limited to settable values. For example, setup screen 150 accepts both setting to black and white and setting to color, but setup screen 185 only accepts setting to black and white and does not accept setting to color. Furthermore, setup screen 150 accepts setting any number of copies, but setup screen 185 does not accept setting a number of copies greater than the upper limit.

[0081] Figure 13 This diagram illustrates an example of a settings screen 190 displaying a restricted mode when the object is processed as a fax. This settings screen 190 is used for setting up fax processing. However, compared to... Figure 12 Similarly, the settings on the settings screen 190 are limited, just like the settings screen 185 shown. In the limited mode, the settings for printing the source information are valid in the initial settings, and the settings for printing the source information are not allowed on the settings screen 190. Moreover, the redialing operation is not allowed on the settings screen 190.

[0082] Thus, based on the results of user authentication and voice recognition, different operation screens are displayed on the display unit 125. After steps S15, S16, or S26, the process proceeds to... Figure 4The process shown in step S19. In step S19, the processor 121 performs processing according to the settings configured by the user. For example, if in Figure 6 After setting the page to color and 10 copies on the setup screen 150, pressing the start button 153 will cause the image forming unit 127 to form 10 color images corresponding to the image data on the paper. Furthermore, if... Figure 12 After the settings screen 185 is set to black and white and 1 copy, when the start button 186 is pressed, a black and white image corresponding to the image data is formed on the paper by the image forming unit 127. In restricted mode, the content of the processing is restricted compared to the case where user authentication is successful.

[0083] According to the above embodiment, when user authentication is successful but voice recognition fails, the user's menu screen 160 is displayed; when user authentication fails but voice recognition succeeds, a restriction mode setting screen is displayed. Therefore, even if the user is not authenticated or their voice is not recognized, the processing can be performed based on the user's authentication result and voice recognition result. Furthermore, even if user authentication fails but voice recognition succeeds, a restriction mode processing setting screen is displayed, allowing the use of restricted settings and the execution of restricted functions within a safe range from an information security perspective. Therefore, even when user authentication accuracy is low, user convenience can be suppressed. Moreover, the object processing operation, which accepts operation words recognized through voice recognition processing, is displayed on the setting screen 150 and the restriction mode setting screen, saving time spent selecting object processing. Furthermore, the setting screen 150 and menu screen 160 contain user-specific information from the user authentication process, thus improving convenience.

[0084] Furthermore, by pre-registering the user's voice input for the operation word and performing speech recognition by comparing it with that voice input, speech words can be recognized even when the user's pronunciation differs significantly from standard pronunciation due to factors such as accent or speech impediment. As a result, speech recognition accuracy is improved. Moreover, since the operation word is freely determined by the user, it prevents other users from impersonating the user and illegally performing voice operations. As a result, information security is enhanced. Furthermore, by displaying the settings screen 150 or menu screen 160 through the user's voice operation, the content of the processing can be confirmed before execution. Thus, since the user must perform the operation before execution, even when speech recognition accuracy is low, it prevents the user from performing processing different from the desired processing.

[0085] Furthermore, the setting screen 150 includes a setting button 151 that indicates the setting values ​​for the copying process preset by the user, so the setting values ​​can be used more easily compared to the case where the setting button 151 is not included. Moreover, when the setting screen 150 includes a setting button that indicates the setting values ​​used by the user in the last copying process, the setting values ​​can be used more easily compared to the case where the setting button is not included.

[0086] 3. Variations

[0087] The above-described embodiment is one example of the present invention. The present invention is not limited to the above-described embodiment. Furthermore, the above-described embodiment can be modified as follows. In this case, two or more of the following modified examples can be used in combination.

[0088] In the above embodiments, when user authentication and voice recognition are successful, the processor 121 may not necessarily display the setting screen 150. For example, in this case, the processor 121 may not display the setting screen 150 and may perform object processing without user operation. For example, regarding routine processing, sometimes the user has preset setting values. When object processing is this type of processing, even if the setting screen 150 is not displayed and the user does not use the operation unit 124, processing can still be performed according to the preset setting values. According to this variation, even if the user does not use the screen, processing is performed simply by uttering the operation word. Furthermore, in this variation, when the user does not perform the work required for object processing, information prompting the work can be displayed on the display unit 125. For example, when object processing is copying, if no original is placed in the image reading unit 126, a message prompting the placement of the original can be displayed on the display unit 125. Whether an original is placed in the image reading unit 126 is determined, for example, using a sensor that detects the original. Based on this variation, we can see the work required for the user to perform the processing.

[0089] In the above embodiment, in addition to user authentication using voice, user authentication using images captured by the camera unit 129 can also be performed. The processor 121 authenticates the user by comparing the user's facial image captured by the camera unit 129 with pre-registered user facial images. Thus, by performing user authentication using images in addition to user authentication using voice, impersonation, such as using the voice of another person pre-recorded in a recorder, can be prevented.

[0090] In the above embodiment, user authentication and voice recognition processing are performed only when the user is positioned in front of the image processing device 120. "In front" refers to the direction in which the user operating the image processing device 120 is standing when viewed from the image processing device 120. In this case, if the user's voice is captured, the camera unit 129 captures an image of the front of the image processing device 120. The processor 121 analyzes, for example, the image captured by the camera unit 129, thereby detecting the user positioned in front of the image processing device 120. Alternatively, the processor 121 can use an infrared sensor to detect the user positioned in front of the image processing device 120. Furthermore, when a user positioned in front of the image processing device 120 is detected, the processor 121 performs user authentication and voice recognition processing. In this case, as described above, in addition to user authentication using voice, user authentication using the image captured by the camera unit 129 can also be performed. On the other hand, when no user positioned in front of the image processing device 120 is detected, the processor 121 does not perform user authentication and voice recognition processing. According to this variation, it is possible to prevent user authentication and speech recognition processing from being performed by voices other than those of users who do not intend to operate the image processing device 120.

[0091] In the above embodiment, even if user authentication fails but voice recognition succeeds, the process can return to step S11 and display the login screen 140. Furthermore, upon returning to step S11, user authentication can be performed using a different authentication method than voice-based authentication. This other authentication method could be, for example, an authentication method using an image of the user captured by the camera unit 129. In this case, the processor 121 performs user authentication by comparing the user's facial image captured by the camera unit 129 with pre-registered user facial images. If user authentication is successful, the processor 121 displays a setting screen 150 on the display unit 125. According to this variation, even if user authentication fails via voice, the image processing device 120 can still be used.

[0092] In the above embodiment, when user authentication fails but voice recognition succeeds, a common menu screen for all users can be displayed instead of the user-specific menu screen 160. The common menu screen for all users does not contain user-specific information, such as selection buttons 164 and 165 and guidance information 166.

[0093] In the above embodiment, when returning to step S11 after the processing in step S14, user authentication can be performed using an authentication method different from the user authentication using voice. In this case, the processor 121 displays the authentication screen used in user authentication based on the other authentication method on the display unit 125. The other authentication method could be, for example, an authentication method using an image of the user captured by the camera unit 129. Furthermore, if voice recognition is successful, and user authentication is successful using the other authentication method, the processor 121 displays a setting screen 150 on the display unit 125. According to this variation, even when user authentication is not performed via voice, processing can still be performed using the image processing device 120.

[0094] In the above embodiments, the operation word is not necessarily the name of the process. For example, the operation word may contain a username. When the operation word is recognized, for example, a user-specific menu screen 160 corresponding to the username is displayed.

[0095] The processing of the image processing system 100 in the above embodiment is illustrative but not limited to this example. For example, at least a portion of the processing performed in the image processing device 120 can be performed in other devices. Furthermore, at least a portion of the processing performed in the server device 110 can be performed in other devices. For example, it is possible to determine whether the image processing device 120 is allowed to perform processing in a restricted mode, instead of using the server device 110. In this case, the server device 110 may not be required. Moreover, the processing of the image processing system 100 in the above embodiment is illustrative; the order can be changed as long as there is no contradiction.

[0096] In the above embodiments, the information processing apparatus of the present invention is not limited to the image processing apparatus 120. The information processing apparatus can be any apparatus that performs processing according to the user's operation, such as a smartphone or personal computer. For example, in a smartphone, when the user speaks the word "email" to process an email, the voice can be used for user authentication and voice recognition. If the user authentication and voice recognition are successful, the email application is launched, and the email screen used in the email operation is displayed.

[0097] In the above embodiments, processor refers to processor in a broad sense, and includes general-purpose processors (such as CPU: Central Processing Unit, etc.) and special-purpose processors (such as GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic devices, etc.).

[0098] Furthermore, the actions of the processor in the above embodiments are not limited to those of a single processor, but can also be performed collaboratively by multiple processors located in physically separate positions. Moreover, the order of the processor's actions is not limited to the order described in the above embodiments and can be appropriately modified.

[0099] This invention can be provided as a program executed in each of the image processing apparatus 120 and the server apparatus 110. Furthermore, the image processing apparatus 120 and the server apparatus 110 are examples of computers according to this invention. The program can be downloaded via communication lines such as the Internet, or it can be provided in a state where it is recorded on a computer-readable recording medium such as magnetic recording media (magnetic tape, magnetic disk, etc.), optical recording media (optical disc, etc.), optical-magnetic recording media, or semiconductor memory.

[0100] The embodiments of the present invention described above are provided for illustrative purposes. Furthermore, these embodiments do not exhaustively encompass the present invention, nor do they limit the invention to the disclosed methods. It will be apparent to those skilled in the art that various modifications and variations will be readily understood. These embodiments were chosen and described to most readily explain the principles and applications of the invention. Thus, those skilled in the art can understand the invention through various modifications that are assumed to be optimized for specific uses of various embodiments. The scope of the invention is defined by the foregoing claims and their equivalents.

Claims

1. An information processing apparatus, characterized by, having a processor, the processor performs the following steps: acquiring a voice of a user; authenticating the user using the voice and recognizing the voice; displaying an operation screen that differs depending on a result of the authentication of the user and a result of the recognition of the voice and that is used in an operation of performing processing on a display section; and when the user is not authenticated using the voice but the voice is recognized, displaying a selection screen including a first operation image and a second operation image on the display section, the first operation image being an image used in an operation of instructing re-authentication of the user, the second operation image being an image used in an operation of instructing an action in a restricted mode in which a setting of the processing is restricted as compared to a case where the user is authenticated.

2. The information processing apparatus according to claim 1, wherein when the user is authenticated, the operation screen differs depending on the result of the recognition of the voice, the operation screen includes a set value of the processing that is set in advance for the authenticated user.

3. The information processing apparatus according to claim 2, wherein when the set value is not set in advance, the operation screen includes a set value that the user used in the processing before.

4. The information processing apparatus according to claim 1, wherein when the user is authenticated, the processor does not display the operation screen and performs the processing even without the operation.

5. The information processing apparatus according to claim 4, wherein when the user does not perform a work that is necessary to perform the processing, the processor displays information that prompts the work on the display section.

6. The information processing apparatus according to claim 1, wherein the processor performs the authentication of the user and the recognition of the voice based on detection of the user who is located in front of the information processing apparatus.

7. The information processing apparatus according to claim 1, wherein when the user is not authenticated using the voice but the voice is recognized, the processor authenticates the user using an image of the user that is captured by an imaging section, when the user is authenticated using the image, the operation screen is displayed.

8. The information processing apparatus according to claim 1, wherein when the user is not authenticated using the voice, the processor displays an authentication screen that is used in authentication of the user by another authentication method on the display section.

9. The information processing apparatus according to claim 8, wherein when the user is authenticated by the other authentication method in a case where the voice is recognized, the processor displays the operation screen on the display section.

10. The information processing apparatus according to claim 1, wherein When the user is authenticated and the voice is recognized, the processor displays a first operation screen used in an operation in which a setting of a process indicated by a result of recognition of the voice is performed on the display unit, When an operation in the limited mode is instructed by using the second operation image, the processor displays a second operation screen used in an operation in which a setting of a process indicated by a result of recognition of the voice is performed on the display unit, the second operation screen being limited in a set value compared with the first operation screen.

11. The information processing apparatus according to claim 1, wherein When the user is not authenticated and the voice is recognized, the processor limits a function compared with when the user is authenticated.

12. A storage medium storing a program for causing a computer to execute: acquiring a voice of a user; authenticating the user using the voice, and recognizing the voice; and displaying an operation screen used in an operation in which a process is performed on a display unit, the operation screen being different depending on a result of authentication of the user and a result of recognition of the voice; and when the user is not authenticated and the voice is recognized using the voice, displaying a selection screen including a first operation image and a second operation image on the display unit, the first operation image being an image used in an operation in which authentication of the user is instructed to be performed again, the second operation image being an image used in an operation in which an action in a limited mode is instructed, the limited mode being a mode in which a setting of the process is limited compared with a case in which the user is authenticated.

13. An information processing method characterized by comprising: including the steps of: acquiring a voice of a user; authenticating the user using the voice and recognizing the voice; and displaying an operation screen used in an operation in which a process is performed on a display unit, the operation screen being different depending on a result of authentication of the user and a result of recognition of the voice; and when the user is not authenticated and the voice is recognized using the voice, displaying a selection screen including a first operation image and a second operation image on the display unit, the first operation image being an image used in an operation in which authentication of the user is instructed to be performed again, the second operation image being an image used in an operation in which an action in a limited mode is instructed, the limited mode being a mode in which a setting of the process is limited compared with a case in which the user is authenticated.

14. A computer program product, characterised in that, including a program for causing a computer to execute: acquiring a voice of a user; authenticating the user using the voice and recognizing the voice; and displaying an operation screen used in an operation in which a process is performed on a display unit, the operation screen being different depending on a result of authentication of the user and a result of recognition of the voice; and when the user is not authenticated and the voice is recognized using the voice, displaying a selection screen including a first operation image and a second operation image on the display unit, the first operation image being an image used in an operation in which authentication of the user is instructed to be performed again, the second operation image being an image used in an operation in which an action in a limited mode is instructed, the limited mode being a mode in which a setting of the process is limited compared with a case in which the user is authenticated. When the voice is recognized without authenticating the user using the voice, a selection screen including a first operation image and a second operation image is displayed on the display section, the first operation image being an image used in an operation indicating re-authentication of the user, the second operation image being an image used in an operation indicating an action in a restricted mode in which the setting of the process is restricted as compared with the case where the user is authenticated.

Images