Image forming apparatus

Abstract

To immediately make a display screen of a display transitioned to a target operation screen on the basis of a voice instruction input by voice.SOLUTION: An image forming apparatus (1) comprises a display (473), a display control unit (101), a microphone (2), a receiving unit (103), and a control unit (100). While the display control unit (101) displays a selection screen on the display (473), when a voice instruction is input to the microphone (2), the display control unit makes a display screen of the display (473) transitioned from the selection screen (SC1) to a setting screen (SC2) for receiving input of a setting value indicated by the voice instruction, and displays the setting value indicated by the voice instruction on the setting screen (SC2).SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an image forming apparatus capable of accepting voice instructions.

Background Art

[0002] Some image forming apparatuses such as copying machines and multifunction peripherals are provided with an operation panel for a user to perform manual input operations. By performing operations via the operation panel, a user can select one function from among a plurality of functions (for example, a copy function, a scanner function, and a facsimile function) or perform settings for the function.

[0003] Various image forming apparatuses capable of accepting voice instructions have been proposed and put into practical use. For example, the following Patent Document 1 describes that the content of a voice instruction is reflected in job settings, and the following Patent Document 2 describes that job history information intended by a user is narrowed down from a plurality of pieces of job history information based on keywords included in the user's voice.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

[0005] When it is desired to transition the operation screen currently displayed on the operation panel to another operation screen, there are cases where many operation screens have to be displayed in the middle of the transition by manual input operations. For example, when the setting screen for the scanner function is displayed on the operation panel and it is desired to display the setting screen for the copy function, the user needs to once display the function selection screen by manual input operation. As described above, if the user cannot easily move to the target operation screen, it is a trouble for the user.

[0006] This invention has been made in view of the above circumstances, and aims to instantly transition the display screen of the display unit to the desired operation screen based on voice commands input via voice input.

[0007] An image forming apparatus according to one aspect of the present invention is an image forming apparatus comprising an image forming unit for forming an image on a recording medium, the apparatus comprising: a display unit; a display control unit which displays a selection screen on the display unit for allowing a user to select a function to be executed from among a plurality of functions that can be executed by the image forming apparatus, and a setting screen on the display unit for receiving input related to the settings of each function selected on the selection screen; an audio input unit which receives voice input from a user; a reception unit which receives voice instructions based on the voice input unit; and a control unit which executes a job according to the content of the voice instructions received by the reception unit, wherein when the display control unit is displaying the selection screen on the display unit and a voice instruction is input to the audio input unit, the display screen of the display unit transitions from the selection screen to the setting screen for receiving input of the setting value indicated by the voice instruction, and the setting screen displays the setting value indicated by the voice instruction.

[0008] According to the present invention, the display screen of the display unit can be instantly transitioned to the desired operation screen based on voice commands input via voice input. [Brief explanation of the drawing]

[0009] [Figure 1] This is a perspective view showing the external appearance of an image forming apparatus according to one embodiment of the present invention. [Figure 2] This is a functional block diagram that schematically shows the main internal components of an image forming apparatus. [Figure 3] (A) and (B) are diagrams showing examples of operation screens displayed on the display unit. [Figure 4] (A) and (B) are diagrams showing examples of operation screens displayed on the display unit. [Figure 5] This figure shows an example of the operation screen displayed on the display unit. [Figure 6] This figure shows an example of the operation screen displayed on the display unit. [Figure 7] This flowchart shows an example of processing performed by the control unit of an image forming apparatus. [Figure 8] This figure shows an example of the operation screen displayed on the display unit. [Modes for carrying out the invention]

[0010] Hereinafter, an image forming apparatus according to one embodiment of the present invention will be described with reference to the drawings. Figure 1 is a perspective view showing the external appearance of an image forming apparatus according to one embodiment of the present invention. Figure 2 is a functional block diagram schematically showing the main internal configuration of the image forming apparatus. The image forming apparatus 1 according to the first embodiment is a multifunction device that combines multiple functions such as a copy function, a printer function, a scanner function, and a facsimile function, and is composed of a document feeding unit 6, a document reading unit 5, an image forming unit 12, a fixing unit 13, a paper feeding unit 14, a storage unit 8, a human presence sensor 21, an operation unit 47, a facsimile communication unit 71, a network interface unit 91, a microphone 22, and a speaker 23.

[0011] The document feeding unit 6 is configured to open and close on the upper surface of the document reading unit 5 by a hinge (not shown), and functions as a document holder cover when reading documents placed on a platen glass (not shown). The document feeding unit 6 is also known as an ADF (Auto Document Feeder) or DP (Document Processor), and is equipped with a document tray 61 that supplies documents placed on the document tray 61 to the document reading unit 5 one at a time.

[0012] The process of reading a document in the image forming apparatus 1 will now be explained. The document reading unit 5 optically reads the image of the document supplied to the document reading unit 5 by the document feeding unit 6, or the document placed on the platen glass, and generates image data. The image data generated by the document reading unit 5 is stored in an image memory (not shown) or the like.

[0013] A case where an image forming operation is performed by the image forming apparatus 1 will be described. Based on image data generated by a document reading operation or image data received from a computer serving as an external device connected to a network (for example, a personal computer), an image forming section 12 forms an image on recording paper as a recording medium fed from a paper feeding section 14.

[0014] A fixing section 13 heats and pressurizes the recording paper on which a toner image has been formed by the image forming section 12 to fix the toner image onto the recording paper, and the recording paper that has undergone the fixing process is discharged to a discharge tray 151. The paper feeding section 14 includes a plurality of paper feeding cassettes 141.

[0015] A storage section 8 is a large-capacity storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various control programs and the like.

[0016] A human detection sensor 21 detects a person approaching the image forming apparatus 1. As the human detection sensor 21, for example, a sensor that detects infrared rays emitted by a human body is used.

[0017] An operation section 47 receives instructions such as an image forming operation execution instruction from an operator for various operations and processes executable by the image forming apparatus 1. The operation section 47 includes a display section 473 that displays operation guidance and the like for the operator. Furthermore, the operation section 47 receives input of instructions from a user via a touch panel provided in the display section 473, based on a user's operation (touch operation) on an operation screen displayed on the display section 473 or a user's operation on physical keys.

[0018] The display section 473 is formed of an LCD (Liquid Crystal Display) or the like. The display section 473 includes a touch panel. When an operator performs an operation of touching buttons or keys displayed on the screen, an instruction associated with the position where the touch operation was performed is received via the touch panel.

[0019] The facsimile communication unit 71 includes an unillustrated encoding / decoding unit, a modulation / demodulation unit, and an NCU (Network Control Unit), and performs facsimile transmission and reception using a public telephone line network or the like.

[0020] The network interface unit 91 is a communication interface that transmits and receives various data to and from an external device (e.g., a personal computer) in a local area or on the Internet.

[0021] The microphone 22 is an audio input unit that collects sounds around the image forming apparatus 1 and converts the collected sounds into electric signals (audio data). The microphone 22 is provided at an appropriate position where it is easy to collect the user's uttered voice, for example, on the operation unit 47.

[0022] The speaker (audio output unit) 23 outputs various sounds including operation sounds and sound effects when operating the operation unit 47, guidance voice explaining the operation method, and warning sounds when a trouble occurs in the image forming apparatus 1. For example, the speaker 23 is provided at a position invisible from the outside of the image forming apparatus 1, for example, inside the operation unit 47.

[0023] The control unit 10 is configured to include a processor, a RAM (Random Access Memory), a ROM (Read Only Memory), and a dedicated hardware circuit. The processor is, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or an MPU (Micro Processing Unit). The control unit 10 includes a control section 100, a display control section 101, a voice analysis section 102, and a reception section 103.

[0024] The control unit 10 functions as a control unit 100, a display control unit 101, a voice analysis unit 102, and a reception unit 103 through operation by the processor in accordance with the control program stored in the memory unit 8. However, these control units 100, etc., can also be configured by hardware circuits, rather than by operation in accordance with the control program by the control unit 10. The same applies to each embodiment below unless otherwise specified.

[0025] The control unit 100 is responsible for the overall operation control of the image forming apparatus 1. The control unit 100 is connected to the document feeding unit 6, document reading unit 5, image forming unit 12, fixing unit 13, paper feeding unit 14, storage unit 8, human presence sensor 21, operation unit 47, facsimile communication unit 71, network interface unit 91, microphone 22, and speaker 23, and performs drive control of each of these units. For example, the control unit 100 controls the operation of the image forming unit 12, etc., to form the document image obtained by reading by the document reading unit 5 onto the recording paper used as a recording medium.

[0026] Furthermore, when the motion sensor 21 detects an approaching person, the control unit 100 controls the microphone 22 to turn it on, enabling voice input to the microphone 22. After a predetermined time (for example, 30 seconds) has elapsed since the motion sensor 21 stopped detecting an approaching person, the control unit 100 controls the microphone 22 to turn it off. Note that the control unit 100 can also switch the microphone 22 on and off according to instructions input to the operation unit 47.

[0027] The display control unit 101 controls the display operation of the display unit 473. For example, the display control unit 101 displays a selection screen on the display unit 473 for the user to select a function to be executed from among several functions that can be executed in the image forming apparatus 1, and also displays a setting screen on the display unit 473 for receiving input related to the settings of each function selected on the selection screen.

[0028] Figures 3(A) and 3(B) show examples of screens displayed on the display unit 473. The operation screen SC1 shown in Figure 3(A) is a selection screen that allows the user to select a function to be executed from among several functions that can be executed in the image forming apparatus 1. Operation screen SC1 displays buttons such as "Copy," "Send (Scanner function)," and "Fax (Facsimile function)." Operation screen SC1, which functions as a selection screen, also serves as the home screen. When the user presses the "Copy" button, the operation unit 47 receives a copy function selection instruction, and the control unit 100 displays the operation screen SC2 shown in Figure 3(B) on the display unit 473 according to this instruction.

[0029] The operation screen SC2 is displayed on the display unit 473 as a lower level than the "Home" screen. The operation screen SC2 is a settings screen for receiving input related to the "Copy" function settings. In the center of the "Copy" function settings screen, six buttons are displayed labeled "Paper Selection," "Color Selection," "Page Collation," "Reduce / Enlarge," "Duplex / Split," and "Staple / Punch." These buttons are images for receiving settings related to the "Copy" function.

[0030] Furthermore, on the SC2 operation screen, below the six buttons mentioned above, there is a button for accepting the specification of a favorite registration setting, which has pre-registered settings that differ from the default values. This button is labeled, for example, "ID Card Copy." "ID Card Copy" is a function used when you want to copy both the front and back of an ID card, such as a driver's license or health insurance card, onto a single sheet of paper. For "ID Card Copy," settings different from the default values ​​are pre-registered as "Copy" function settings, such as "A6 (Paper Selection)," "Color (Color Selection)," "2in1 (Page Collation)," "A6 (Document Size)," "Do not set (Continuous Scanning)," "Set (Erase Outside Document Area)," and "Outer Frame: 1mm (Frame Eraser)." Therefore, when the user presses the "ID Card Copy" button, the operation unit 47 receives an instruction to reflect these different default settings in the copy function, and this instruction is then received by the reception unit 103.

[0031] The voice analysis unit 102 converts the electrical signals (voice data) converted by the microphone 22 into text data using existing voice recognition technology, and then analyzes the text data using existing natural language processing technology to recognize voice instructions from the user.

[0032] The reception unit 103 receives instructions from the user via manual input through the operation unit 47 (including the touch panel) on the screen currently displayed on the display unit 473, and voice instructions (analysis results by the voice analysis unit 102) recognized by the voice analysis unit 102 via the microphone 22. The control unit 100 executes a job according to the content of the instructions received by the reception unit 103.

[0033] The display control unit 101 displays a settings screen on the display unit 473 that shows the content indicated by the instructions entered by manual input or the voice instructions. For example, when the operation screen SC2, which is a settings screen as shown in Figure 3(B), is displayed on the display unit 473, the user inputs instructions via the operation unit 47 to set "Paper Selection" to "A4", "Color Selection" to "Black and White", and "Page Consolidation" to "2in1", and the reception unit 103 receives these instructions. At this time, the display control unit 101 displays the operation screen SC2 (settings screen) on the display unit 473, as shown in Figure 4(A), which switches the "Paper Selection" setting from "Automatic" to "A4", the "Color Selection" setting from "Full Color" to "Black and White", and "Page Consolidation" from "Off" to "2in1".

[0034] Furthermore, when the operation screen SC2 shown in Figure 3(B) is displayed on the display unit 473, and the user speaks the keywords "A4", "black and white", and "2in1", and the reception unit 103 receives these voice commands via the speaker 23 and the voice analysis unit 102, the display control unit 101 displays the operation screen SC2 shown in Figure 4(B) on the display unit 473, with the "Paper Selection" setting changed from "Automatic" to "A4", the "Color Selection" setting changed from "Full Color" to "Black and White", and "Page Consolidation" changed from "Off" to "2in1". The operation screen SC2 shown in Figure 4(B) differs from the operation screen SC2 shown in Figure 4(A) in that it displays an operation button B1 labeled "Voice Input Item".

[0035] On the operation screen SC2 shown in Figure 4(B), when an item output instruction is received by the reception unit 103 via the operation unit 47 by pressing the operation button B1, the control unit 100 controls the voice output operation of the speaker 23 to output the content of the setting item set based on the voice instruction as voice. For example, the speaker 23 will output the voice message, "A4, black and white, and 2-in-1 have been set by voice." The reception unit 103 also accepts settings for setting items not displayed on the operation screen (setting screen) via voice instructions through the speaker 23 and the voice analysis unit 102. The control unit 100 also outputs the content of setting items not displayed on the operation screen, which have been set based on the voice instruction, as voice from the speaker 23.

[0036] Unlike manual input, voice commands may have lower accuracy in receiving instructions. Therefore, users may feel uneasy about whether the settings have been correctly received via voice commands. This is especially true for settings that are hidden and not displayed on the display unit 473. By outputting the settings made via voice commands as audio, users can confirm the settings made via voice commands, thus alleviating the aforementioned anxiety.

[0037] Furthermore, the display control unit 101 may, when a user manually inputs a confirmation instruction (output instruction) into the operation unit 47 and it is received by the reception unit 103, display the contents of each setting value that has been set based on the voice instruction received by the reception unit 103 on the display unit 473. When a confirmation instruction is input via voice instruction through the microphone 22 and the voice analysis unit 102 and it is received by the reception unit 103, the control unit 100 may output the contents of each setting value that has been set based on the voice instruction by voice.

[0038] When the operation screen SC1 shown in Figure 3(A) is displayed on the display unit 473, if the user utters the keyword "black and white copy", the voice instruction to set the value to "black and white copy" is received by the reception unit 103 via the speaker 23 and the voice analysis unit 102. In other words, when the display control unit 101 is displaying the operation screen SC1, which is a selection screen, on the display unit 473, and a voice instruction is input to the speaker 23 and received by the reception unit 103 via the voice analysis unit 102, the display control unit 101 transitions the display screen of the display unit 473 from the operation screen SC1, which is a selection screen, to the operation screen SC2, which is a setting screen for the copy function that accepts the input of the setting value indicated by the voice instruction. Furthermore, as shown in the example in Figure 5, the display control unit 101 displays the setting value indicated by the received voice instruction, in this case "black and white copy", on the operation screen SC2. Furthermore, information regarding which function each setting screen corresponds to and what kind of setting values ​​it accepts is pre-stored in memory built into the control unit 10, for example, through an operation based on the control program. Based on this information, the display control unit 101 identifies the setting screen that accepts the setting values ​​indicated by the voice command.

[0039] However, when the operation screen SC1, which serves as a selection screen as shown in Figure 3(A), is displayed on the display unit 473, if the user utters the keyword "black and white" and the voice command is received by the reception unit 103, the "black and white" indicated by the voice command is common to multiple functions, including the copy function, scanner (transmission) function, and facsimile (fax) function. Therefore, it is not possible to narrow down the setting screen for receiving the setting value indicated by the voice command to a single screen.

[0040] Therefore, if the setting value indicated by the voice command is common to multiple functions and it is not possible to narrow down the setting screen for accepting input of the setting value indicated by the voice command to a single screen, the display control unit 101 will not transition from the selection screen to the setting screen, and on the selection screen, it will hide the images that indicate functions in which the setting value indicated by the voice command is not used, and will only display the images that indicate each function in which the setting value indicated by the voice command is used.

[0041] For example, if a user speaks the keyword "2in1" and the voice command is received by the reception unit 103, it is not possible to narrow down the setting screen for receiving the setting value indicated by the voice command to just one, but it is possible to narrow it down to two: the copy function and the scanner function. For this reason, the display control unit 101 displays an operation screen SC3 on the display unit 473 that shows only the image (button) indicating the copy function and the image (button) indicating the scanner function, as shown in Figure 6. In other words, the display control unit 101 displays an operation screen SC3 (selection screen) on the display unit 473 that does not display the images (buttons) indicating functions that are not expected to be desired by the user, such as "fax," "user box," "system settings," and "internet."

[0042] Furthermore, the reception unit 103 also applies the setting values ​​indicated by the voice command as the setting values ​​for the pre-registered favorite registration settings mentioned above. For example, when the operation screen SC2 is displayed, if the user utters the keyword "ID card copy" and the voice command is received by the reception unit 103 via the voice analysis unit 102 or by operating the operation unit 47, the reception unit 103 changes each setting value that is different from the default in the favorite registration settings to the setting values ​​that have already been received by the voice command at that point and accepts it.

[0043] Next, an example of processing performed by the control unit 10 of the image forming apparatus 1 will be explained based on the flowchart shown in Figure 7. This processing is performed when the audio analysis unit 102 analyzes the electrical signal (audio data) converted by the microphone 22.

[0044] When the display control unit 101 is displaying the operation screen SC1, which is a selection screen, on the display unit 473, and a voice command is input to the speaker 23 and the voice command is received by the reception unit 103 via the voice analysis unit 102, the display control unit 101 identifies a setting screen (a function to which the setting value indicated by the voice command is applied) for receiving input of the setting value indicated by the voice command (S1).

[0045] Furthermore, the display control unit 101 determines whether there is only one such identified setting screen (function) (S2). If the display control unit 101 determines that there is only one identified setting screen (function) (YES in S2), the display control unit 101 displays the identified setting screen on the display unit 473 (S3). After this, the process ends.

[0046] On the other hand, if the display control unit 101 determines that there is more than one identified setting screen (function) (NO in S2), the display control unit 101 does not transition from the selection screen to the setting screen. Instead, on the selection screen, it hides the images representing functions that do not use the setting value indicated by the voice instruction, and displays only the images representing each function that does use the setting value indicated by the voice instruction (S4). After this, the process ends.

[0047] Thus, in this embodiment, even when a setting value common to multiple functions is entered by voice, the user is guided by a display focused on the function to which the voice-entered setting value applies. Therefore, the user can more easily select the setting screen necessary to enter the setting value they want to set.

[0048] Furthermore, when the display control unit 101 acquires operating status information indicating that the operating mechanism of the image forming apparatus 1 has been operated, it may transition the display screen of the display unit 473 to a setting screen for accepting input of setting values ​​for a function that is pre-associated with the content of the operating status information, among a plurality of functions that use the setting values ​​indicated by the voice command.

[0049] For example, when the functions for which the setting values ​​indicated by the voice command are used are multiple copy and scanner functions, the display control unit 101 acquires open / close detection information from an open / close switch (not shown) that detects the opening and closing of the document feed unit 6. If the open / close detection information indicates "open," the display unit 473 displays a setting screen for accepting input of the setting value of the scanner function that is pre-associated with the "open" information. If the open / close detection information indicates "closed," the display unit 473 displays a setting screen for accepting input of the setting value of the copy function that is pre-associated with the "closed" information.

[0050] Furthermore, the control unit 100 may output the contents of the setting value indicated by the voice instruction to the speaker 23 each time the voice instruction is received by the reception unit 103. Alternatively, when the operation screen SC2, which serves as a setting screen for displaying the setting value indicated by the voice instruction, is displayed, the control unit 100 may output the contents of the setting values ​​indicated by the voice instruction that are the target of the display to the speaker 23 all at once.

[0051] Furthermore, if the settings indicated by the above voice instructions are output to speaker 23 in this manner, if the number of setting values ​​outputted by voice becomes too large, the user may actually find it more difficult to understand the content of the settings.

[0052] Therefore, in a further embodiment, if the number of setting values ​​to be output by voice is greater than or equal to a predetermined number (for example, 5), the control unit 100 will not output the above content by voice, but will instead output a predetermined warning message by voice to the speaker 23. For example, the control unit 100 will output the voice message "Please check the setting values ​​on the screen" via the speaker 23.

[0053] Furthermore, in a further embodiment, the display control unit 101 may display a preview image on the display unit 473, assuming that the control unit 100 has executed the job by applying the setting values ​​indicated by the voice command on the setting screen. This allows the user to confirm whether the setting values ​​indicated by the voice command have been accurately received and to see the expected result when those settings are applied, as illustrated in the example in Figure 8, before executing the job.

[0054] Up to this point, we have described the case in which the audio data collected and converted by the microphone 22 is analyzed by the audio analysis unit 102 in the image forming apparatus 1. However, as a further embodiment, a speaker with a built-in microphone and AI (artificial intelligence) assistant function, known as a smart speaker, may be used to collect the user's voice, and the image forming apparatus 1 may use the speaker as an audio input unit and have the reception unit 103 receive the audio analysis results from the speaker as set values.

[0055] The present invention is not limited to the configuration of the above embodiments and various modifications are possible. Furthermore, the configuration and processing shown in the above embodiments using Figures 1 to 8 are merely one embodiment of the present invention and are not intended to limit the present invention to such configuration and processing.

Claims

1. An image forming apparatus comprising an image forming unit for forming an image on a recording medium, Display unit and A display control unit that displays a selection screen on the display unit for the user to select a function to be executed from among a plurality of functions that can be executed by the image forming apparatus, and displays a setting screen on the display unit for receiving input related to the settings of each function selected on the selection screen, A voice input unit where the user's voice is input, A receiving unit that receives voice instructions based on the voice input to the voice input unit, The system comprises a control unit that executes a job according to the content of the voice instructions received by the reception unit, The display control unit displays the setting value indicated by the voice command on the setting screen, and also displays operation buttons for outputting the content of the setting value indicated by the voice command. An operating unit into which instructions are input by the user, It further includes an audio output section, When the control unit receives an input operation for the operation button via the operation unit, it causes the content of the setting value indicated by the voice instruction to be output to the voice output unit. The reception unit also accepts settings for items not displayed on the settings screen, based on voice instructions entered into the voice input unit. The control unit also outputs, via the voice output unit, the settings for setting items that are set based on the voice instructions but are not displayed on the setting screen.

2. An image forming apparatus comprising an image forming unit for forming an image on a recording medium, Display unit and A display control unit that displays a selection screen on the display unit for the user to select a function to be executed from among a plurality of functions that can be executed by the image forming apparatus, and displays a setting screen on the display unit for receiving input related to the settings of each function selected on the selection screen, A voice input unit where the user's voice is input, A receiving unit that receives voice instructions based on the voice input to the voice input unit, The system comprises a control unit that executes a job according to the content of the voice instructions received by the reception unit, The display control unit displays the setting value indicated by the voice command on the setting screen, and also displays operation buttons for outputting the content of the setting value indicated by the voice command. An operating unit into which instructions are input by the user, It further includes an audio output section, When the control unit receives an input operation for the operation button via the operation unit, it causes the content of the setting value indicated by the voice instruction to be output to the voice output unit. The control unit, when the number of setting values ​​indicated by the voice instruction is greater than or equal to a predetermined number, does not output the content of the setting values ​​indicated by the voice instruction to the voice output unit, but instead outputs a predetermined warning message to the voice output unit, in an image forming apparatus.

3. The image forming apparatus according to claim 1, wherein when the display control unit receives an input operation for the operation button via the operation unit, it causes the content of the set value indicated by the voice instruction to be displayed on the display unit.

4. The image forming apparatus according to claim 1, wherein the display control unit causes the display unit to display a preview image of the job assuming that the job was executed by applying the setting value indicated by the voice instruction on the setting screen.

Images