Image processing device

Abstract

Provided is an image processing device capable of operation instruction given by a voice via a voice device having a voice notification function, the image processing device comprising an image processing unit for performing image processing, and a display unit for displaying information related to the image processing. The display unit displays a content image to be subjected to the image processing based on the operation instruction given by the voice.

Description

Image processing device

[0001] The present disclosure relates to an image processing device.

[0002] In recent years, voice assistant systems that receive operational instructions given by a user through voice and cause various devices to operate in accordance with the operational instructions have become increasingly popular. In such voice assistant systems, a voice including an operational instruction given by the user is picked up by an audio device such as a smart speaker, converted into audio information, and transmitted to a server. The server then analyzes the audio information, extracts the operational instruction contained in the audio information, and transmits the operational instruction information corresponding to the extracted operational instruction to a target device, thereby causing the device to operate.

[0003] Such control of operation by a voice assistant system is also applied to image processing devices such as printers. For example, Patent Document 1 describes an invention in which a user can use a printing service by issuing a print instruction to a printer via a smart speaker. In this invention, if there are options related to the user's print instruction, such as multiple templates that can be used when printing text, the smart speaker notifies the user of the options by voice, and the user speaks the selected option, causing a server to receive information about the option via the smart speaker, and the server then transmits operation instruction information corresponding to the option to the printer.

[0004] Japanese Patent Application Laid-Open No. 2021-163163

[0005] According to a first aspect, an image processing device is an image processing device capable of receiving operational instructions by voice via an audio device having a voice notification function, and includes an image processing unit that performs image processing, and a display unit that displays information related to the image processing, wherein the display unit displays a content image that is the target of the image processing based on the operational instruction by voice.

[0006] FIG. 1 is a schematic diagram showing the configuration of an image processing system to which an image processing device according to an embodiment is applied. FIG. 2 is a front view of the image processing device according to an embodiment. FIG. 3 is a functional block diagram of the image processing device according to an embodiment. FIG. 4 is a processing flow diagram of the image processing system to which an image processing device according to an embodiment is applied. FIG. 5 is a schematic diagram showing character information displayed on a display unit of an image processing device according to an embodiment. FIG. 6A is a schematic diagram showing how character information is scrollably displayed on a display unit of an image processing device according to an embodiment, where the beginning of the character information is displayed and FIG. 6B is a schematic diagram showing how the latter half of the character information is displayed. FIG. 7A is a schematic diagram showing how a two-dimensional code is displayed together with character information on a display unit of an image processing device according to an embodiment. FIG. 7B is a schematic diagram showing how detailed content of the character information corresponding to the two-dimensional code is displayed on a smartphone that has read the two-dimensional code of FIG. 7A. FIG. 8A is a schematic diagram showing how only a two-dimensional code is displayed on a display unit of an image processing device according to an embodiment. FIG. 8B is a schematic diagram showing how character information corresponding to the two-dimensional code is displayed on a smartphone that has read the two-dimensional code of FIG. 8A. FIG. 9A is a schematic diagram showing how selectable text information is displayed on the display unit of the image processing device according to the embodiment, and FIG. 9B is a schematic diagram showing how related information related to the selected text information is displayed on the display unit. FIG. 10 is a schematic diagram showing how user answer candidates for a voice notification are displayed on the display unit of the image processing device according to the embodiment. FIG. 11 is a schematic diagram showing how a content image is displayed on the display unit of the image processing device according to the embodiment. FIG. 12 is a schematic diagram showing how a changed content image is displayed on the display unit of the image processing device according to the embodiment. FIG. 13 is a schematic diagram showing how the magnification of a content image is changed by a pinch-out operation. FIG. 14 is a schematic diagram showing how the content image is changed by a drag operation. FIG. 15 is a schematic diagram showing how a print preview image is displayed on the display unit of the image processing device according to the embodiment. FIG. 16 is a diagram for explaining the operation of the image processing device and the audio device using an example in which the content image is a coloring book.Fig. 17 is a diagram for explaining the operation of the image processing device and the audio device, taking the case where the content image is a coloring book as an example. Fig. 18 is a functional block diagram of a server according to a modified example.

[0007] (Summary of the embodiment) Information such as selectable options presented by a smart speaker via voice may be difficult to hear due to ambient noise, or may be missed while concentrating on other tasks, making it difficult to accurately understand the information presented by the smart speaker.

[0008] Therefore, the present disclosure provides an image processing device that allows a user to accurately understand information presented by an audio device without failing to receive it.

[0009] The image processing device of the embodiment is an image processing device capable of operating in conjunction with an audio device having audio notification and communication functions, and is equipped with an image processing unit that performs image processing, a display unit that displays information related to the image processing, and a control unit that controls the operation of the image processing unit and the display unit, and the control unit displays, on the display unit, text information corresponding to the audio notification regarding the image processing by the audio device and / or an information code that can be converted into text information corresponding to the audio notification, in synchronization with the audio notification.

[0010] In the image processing device of this embodiment, the display unit of the image processing device displays text information corresponding to the voice notification from the audio device in synchronization with the voice notification, allowing information related to printing operations to be recognized and accurately understood both visually and audibly.

[0011] The image processing apparatus according to the embodiment will be described below.

[0012] (Configuration of Image Processing System) FIG. 1 is a schematic diagram showing the configuration of an image processing system 1 to which an image processing device 2 according to an embodiment is applied.

[0013] 1, the image processing system 1 includes an image processing device 2 and an external audio device (hereinafter also simply referred to as an "audio device") 3 used together with the image processing device 2. The image processing device 2 and the audio device 3 are connected to a network N via an access point 4, and are connected to a server 5 via the network N.

[0014] In this embodiment, the image processing device 2 can operate in cooperation with the audio device 3. Specifically, the image processing device 2 can receive audio operation instructions via the audio device 3, which has an audio notification function. The image processing device 2 has a display function for text information corresponding to audio notifications from the audio device 3, and a printer function capable of printing images. However, this embodiment is not limited to this, and the image processing device 2 may be a device having various functions other than a printer function, such as a scanner function for reading and saving images as data, a copy function for printing the read images using a printer mechanism, or a facsimile function for transmitting the read images to another device, or may be a device without a printer function.

[0015] The audio device 3 collects audio including operational instructions to various devices uttered by the user, converts the audio into audio information, and transmits the audio information to the server 5. The audio device 3 also receives the audio information transmitted from the server 5 and produces sound. The audio device 3 is a device having sound collection, sound production, and communication functions, such as a smart speaker, smartphone, smart watch, or tablet terminal.

[0016] The access point 4 is a device for connecting a plurality of devices to each other and to the network N. The access point 4 is installed, for example, in the user's home.

[0017] The server 5 analyzes the audio information received from the audio device 3 via the network N, extracts the operation instructions contained in the audio information, and transmits operation instruction information corresponding to the extracted operation instructions to the target device such as the image processing device 2, thereby causing the device to perform the operation.

[0018] (Configuration of image processing device) Fig. 2 is a front view of the image processing device according to the embodiment. Fig. 3 is a functional block diagram of the image processing device according to the embodiment. As shown in Figs. 2 and 3, the image processing device 2 is capable of operating in cooperation with the audio device 3, and is configured to include an image processing unit 20, a display unit 30, an input unit 40, a communication unit 50, a control unit 60, and a storage unit 70.

[0019] In this embodiment, the image processing unit 20 is a printer mechanism having an image printing function. Under the control of the control unit 60, the image processing unit 20 prints an image based on data received from the outside by the communication unit 50, an image based on data stored in a recording medium (not shown) connected to the image processing device 2, or an image based on data stored in the storage unit 70.

[0020] In this embodiment, the image processing unit 20 is a printer mechanism having an image printing function, but this is not limited to this. The image processing unit 20 may also have various functions other than a printer mechanism, such as a scanner function that reads an image and stores it as data in the memory unit 70, or a copy function that prints the read image using a printer mechanism. The display unit 30 displays text information and / or information codes related to the printing function. The information code may be a one-dimensional code such as a barcode or a two-dimensional code such as a QR (Quick Response) Code (registered trademark). The display unit 30 also displays various information such as information related to the settings and adjustments of the image processing device 2, as well as images related thereto. In this embodiment, the display unit 30 may be a touch panel display formed by overlapping a display and a touch panel, and may perform part of the functions of the input unit 40. The display unit 30 is configured as a liquid crystal display or an organic electroluminescence (EL) display. These displays have a relatively large number of regularly arranged pixels and can display images including arbitrary shapes based on image data. The display unit 30 may be capable of displaying color images.

[0021] The input unit 40 is configured to accept input operations from a user, and is composed of the display unit 30 as a touch panel display and one or more physical buttons 40b, as shown in Fig. 2. When a touch operation is performed on the display unit 30, the input unit 40 detects the position of the touch operation and outputs the detection result to the control unit 60. The input unit 40 may include an A / D converter that performs analog-to-digital (A / D) conversion of the output of the display unit 30.

[0022] The communication unit 50 performs image data communication under the control of the control unit 60. Specifically, the communication unit 50 realizes communication between the image processing device 2 and other devices. In this embodiment, the other devices include the audio device 3, a personal computer (PC), a mobile terminal (smartphone, etc.), another image processing device, and a server 5. Communication may be performed directly with the other devices or indirectly via a network N. Examples of the network N include a telephone network, the Internet, a private network, and a LAN (Local Area Network). Furthermore, communication may be wired communication or wireless communication.

[0023] The control unit 60 includes one or more processors and controls the entire image processing device 2. The control unit 60 executes various processes by executing control programs stored in the storage unit 70. The control unit 60 controls the operations of the image processing unit 20, the display unit 30, the input unit 40, and the communication unit 50. For example, when the control unit 60 receives an operation (instruction) from a user via the input unit 40, it executes a process according to the operation content. The input unit 40 displays various operation screens on the display unit 30 in accordance with the instructions from the control unit 60. The control unit 60 may include a logic circuit or the like configured to perform only certain operations.

[0024] The storage unit 70 includes various memories such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an auxiliary storage device. The combination of the control unit 60 and the storage unit 70 may be regarded as a computer. The programs executed by the control unit 60 are stored in the ROM of the storage unit 70 and / or the auxiliary storage device, for example.

[0025] The image processing unit 20, the display unit 30, the input unit 40, the communication unit 50, the control unit 60, and the storage unit 70 are connected by, for example, a bus. In Fig. 3, all the components are connected to a single bus BS, but multiple buses may be provided. For example, an address bus, a data bus, and a control bus may be provided.

[0026] (Example of Operation of Image Processing System) Next, a description will be given of the operation of the image processing system 1 having the above-described configuration. Fig. 4 is a process flow diagram of the image processing system 1 to which the image processing device 2 according to the embodiment is applied.

[0027] As shown in FIG. 4, in the image processing system 1, the audio device 3 first picks up the voice uttered by the user, including an operation instruction to the image processing device 2, converts it into audio information (step S1), and transmits the audio information to the server 5 (step S2).

[0028] Next, the server 5 analyzes the received voice information and extracts the operation instruction contained in the voice information (step S3).

[0029] The server 5 also transmits notification information (voice notification information) to the voice device 3 to cause the voice device 3 to issue a voice notification related to the operation instruction (step S4).

[0030] Furthermore, the server 5 transmits notification information (text notification information) for displaying text information related to the operation instruction to the image processing device 2 simultaneously with or after a predetermined time from the voice notification to the audio device 3 (step S5). The text notification information may include information (e.g., two-dimensional code information) for displaying an information code (e.g., two-dimensional code) corresponding to the text notification information on the image processing device 2. Furthermore, the server 5 may transmit only two-dimensional code information to the image processing device 2 instead of the text notification information.

[0031] If the operation instruction extracted in step S3 can be immediately executed, the server 5 transmits operation instruction information corresponding to the operation instruction to the image processing device 2 and executes steps S4 and S5. On the other hand, if it is necessary to give a predetermined notice to the user before executing the operation instruction extracted in step S3, for example, when there are multiple options for the operation instruction, the server 5 executes steps S4 and S5 without transmitting operation instruction information to the image processing device 2.

[0032] Next, the audio device 3 performs audio notification based on the received audio notification information (step S6).

[0033] Furthermore, the image processing device 2 displays text information corresponding to the voice notification on the display unit 30 based on the received text notification information (step S7). The text information is a text version of the voice included in the voice notification.

[0034] When the text notification information includes information to be displayed on the image processing device 2 (for example, two-dimensional code information), the image processing device 2 displays, for example, a two-dimensional code together with the text information on the display unit 30. Furthermore, when the image processing device 2 receives, for example, only two-dimensional code information, it displays the two-dimensional code on the display unit 30.

[0035] The display of text information (and / or information code) on the display unit 30 by the image processing device 2 is performed in synchronization with the voice notification by the voice device 3 under the control of the control unit 60. Here, the synchronization of text information and voice notification is a concept including the fact that at least a part of the display time of the text information is included in the execution time of the voice notification by the voice device 3, and that the display time of the text information and the execution time of the voice notification are continuous.

[0036] Specifically, synchronization includes (i) the display time of the text information and the execution time of the voice notification being the same, (ii) the display time of the text information and the execution time of the voice notification partially overlapping, and (iii) the display time of the text information and the execution time of the voice notification being different times but occurring consecutively.

[0037] In the case of (ii) of synchronization, the display of the text information may be performed simultaneously with the start of the execution of the voice notification, or may be started after a predetermined time has elapsed since the start of the execution of the voice notification and before the end of the voice notification. Furthermore, in the case of (ii) above, the display of the text information may be started before the voice notification. Furthermore, in the case of (ii) above, the display of the text information may be completed within the execution time of the voice notification, or may be completed after a predetermined time has elapsed after the execution time of the voice notification has elapsed.

[0038] The display time of the character information (and / or information code) by the image processing device 2 is determined based on information (display time information) related to the display time included in the character notification information (and / or, for example, two-dimensional code information) from the server 5. The display time information may include only the duration (for example, 20 seconds) of the display time of the character information (and / or information code), so that the image processing device 2 displays the character information (and / or information code) for only that duration immediately upon receiving the character notification information (and / or, for example, two-dimensional code information) from the server 5. Alternatively, the display time information may include the start and end times of the display, so that the image processing device 2 displays the character information (and / or information code) based on the start and end times.

[0039] (Example of Display by Image Processing Device) Next, a description will be given of display of characters and / or information codes (two-dimensional codes, etc.) by the image processing device 2. Fig. 5 is a schematic diagram showing character information T displayed on the display unit 30 of the image processing device 2 according to the embodiment.

[0040] As shown in Fig. 5, text information corresponding to the audio notification by the audio device 3 is displayed on the display unit 30 of the image processing device 2. In the example shown in Fig. 5, the audio device 3 issues a voice saying "Printing a coloring book" as the audio notification, and the display unit 30 of the image processing device 2 displays text information T saying "Printing a coloring book" corresponding to the audio notification.

[0041] Figure 6 (A) is a schematic diagram showing how character information T is displayed in a scrollable manner on the display unit 30 of the image processing device 2 of the embodiment, with the first part Ta of the character information T being displayed, and Figure 6 (B) is a schematic diagram showing how the second part Tb of the character information T is displayed.

[0042] The character information T may be displayed in its entirety on the display unit 30 at the same time, or if the display unit 30 is small and the entire character information T cannot fit on the display unit 30, the character information T may be displayed in a scrollable manner, as shown in Figures 6(A) and 6(B). In this case, for example, a scroll bar SB is displayed in an operable manner near the right end of the display unit 30, and initially only the beginning portion Ta is displayed, with the subsequent latter portion Tb hidden. The user can then touch and slide the scroll bar SB to scroll the character information T from the beginning portion Ta to the latter portion Tb, or vice versa, thereby gradually switching the display.

[0043] By displaying the character information T in a scrollable manner in this way, it is possible to deal with cases where the entire character information T does not fit on the display unit 30 .

[0044] Furthermore, if the entire text information T does not fit on the display unit 30, a page-switching display, in which the display switches page by page, may be used instead of a scrolling display. Specifically, initially, only the page of the beginning portion Ta is displayed, and the pages of the latter portion Tb are hidden. Then, when the user touches and slides the scroll bar SB, the display switches from the page of the beginning portion Ta to the page of the latter portion Tb.

[0045] By displaying the text information T in a page-switchable manner in this way, it is possible to deal with cases where the entire text information T does not fit on the display unit 30 .

[0046] Figure 7 (A) is a schematic diagram showing how a two-dimensional code B is displayed together with character information T on the display unit 30 of the image processing device 2 according to the embodiment, and Figure 7 (B) is a schematic diagram showing how detailed content TS of the character information T corresponding to the two-dimensional code B is displayed on a smartphone S that has read the two-dimensional code B in Figure 7 (A).

[0047] 7A, the content of the text information T may be simplified and a two-dimensional code B may be displayed together with the text information T. In this example, the text information T is simplified and displayed on the display unit 30 as "Which would you like to print, a coloring book for adults or a coloring book for children?", along with "...", which indicates that the previous content has been omitted.

[0048] Then, when the user reads this two-dimensional code B, for example, with the camera of the smartphone S, detailed content TS of the text information T may be displayed on the screen DS of the smartphone S.

[0049] Here, as the detailed content TS, the content omitted by "..." in Figure 7 (A) is reflected, and the following text is displayed on the display unit 30: "There are coloring books for adults and coloring books for children. Adult coloring books have complex designs that can be enjoyed for a long time. Adult coloring books include landscapes and geometric patterns. Children's coloring books have simple designs that can be enjoyed in a short time. Children's coloring books include vehicles and characters. Which would you like to print, the adult coloring book or the child's coloring book?"

[0050] In this manner, even by displaying the two-dimensional code B together with the character information T, it is possible to deal with cases where the detailed content TS of the character information T cannot all fit on the display unit 30 .

[0051] Figure 8 (A) is a schematic diagram showing how only a two-dimensional code B is displayed on the display unit 30 of an image processing device 2 according to an embodiment, and Figure 8 (B) is a schematic diagram showing how text information T corresponding to the two-dimensional code B is displayed on a smartphone S that has read the two-dimensional code B in Figure 8 (A).

[0052] As shown in Figure 8 (A), if the display unit 30 of the image processing device 2 is small and it is difficult to display the content of the character information T in a sufficiently identifiable manner, only the two-dimensional code B may be displayed on the display unit 30.

[0053] Then, when the user reads the two-dimensional code B with the camera of the smartphone S, text information T may be displayed on the screen DS of the smartphone S. Here, as the text information T corresponding to the two-dimensional code B, the following text is displayed on the display unit 30: "There are coloring books for adults and coloring books for children. Adult coloring books have complex designs that can be enjoyed for a long time. Adult coloring books include landscapes and geometric patterns. Children's coloring books have simple designs that can be enjoyed in a short time. Children's coloring books include vehicles and characters. Which would you like to print, the adult coloring book or the child's coloring book?"

[0054] In this way, by displaying only the two-dimensional code B on the display unit 30 and displaying the text information T corresponding to the two-dimensional code B by reading it using an external smartphone S or the like, it becomes possible to accommodate even cases where the display unit 30 is small.

[0055] Figure 9 (A) is a schematic diagram showing how selectable character information TC is displayed on the display unit 30 of the image processing device 2 of the embodiment, and Figure 9 (B) is a schematic diagram showing how related information T2 related to the selected character information TC is displayed on the display unit 30.

[0056] As shown in Fig. 9A, character information T displayed on the display unit 30, which is a touch panel display, may include character information TC that can be selected by a touch operation. In Fig. 9A, character information T saying "Print a coloring book" is displayed, and the "coloring book" part of this character information T is selectable character information TC.

[0057] To make it easier for the user to recognize that the selectable character information TC is selectable, it is preferable that the selectable character information TC be displayed in a manner different from that of character information that is not selectable character information TC, for example by changing the color of the background part of the selectable character information TC from that of the other character parts, or by changing the color or thickness of the font, or by displaying the font in a blinking manner.

[0058] Furthermore, the selectable character information TC is not limited to words contained in the character information, but may be at least one of a sentence and a paragraph, and the selectable character information TC can be displayed in various forms.

[0059] When the user touches selectable text information TC, the operation information is transmitted from the image processing device 2 to the server 5. The server 5 generates text notification information for displaying related information T2 related to the received operation information and transmits it to the image processing device 2, and also generates and transmits audio notification information corresponding to the text notification information to the audio device 3.

[0060] The control unit 60 of the image processing device 2 that has received the text notification information causes the display unit 30 to display related information T2 corresponding to the text notification information.

[0061] In the example shown in FIG. 9(B), the display unit 30 of the image processing device 2 that has received the text notification information displays the following related information T2: "There are four types of 'coloring books': Landscapes of cities and mountain villages, etc.; Plants such as flowers and vegetables; Animals such as insects and birds; and Complex geometric patterns."

[0062] The audio device 3 also issues an audio notification corresponding to the related information T2.

[0063] In addition, the related information T2 is not limited to information that embodies the content of the selectable character information TC, but may also include any character information that is different from the selectable character information TC but is associated with the selectable character information TC.

[0064] In this way, by including selectable character information TC in character information T, the amount of information displayed on display unit 30 at one time can be reduced, making it possible to accommodate small display units 30 and presenting information to the user in an easy-to-understand manner.

[0065] FIG. 10 is a schematic diagram showing how candidate answers TB1 and TB2 of a user to a voice notification are displayed on the display unit 30 of the image processing device 2 according to the embodiment.

[0066] As shown in Figure 10, when the voice notification by the voice device 3 includes multiple options, the display unit 30 of the image processing device 2 may display user answer candidates TB1 and TB2 for the options in the text information T.

[0067] 10, the audio device 3 issues a voice notification asking, "There are coloring books for adults and children. Which one would you like to print?" The display unit 30 of the image processing device 2 displays, as text information T corresponding to this voice notification, "Which coloring book would you like to print?", along with two objects (answer candidates TB1 and TB2) as answer candidates: "For adults" indicating a coloring book for adults and "For children" indicating a coloring book for children.

[0068] Then, when the user selects one of the two answer candidates TB1, TB2 by operating the display unit 30, which is a touch panel display, or by voice input via the audio device 3, selection information corresponding to the selected answer candidate TB1, TB2 is transmitted from the image processing device 2 or the audio device 3 to the server 5.

[0069] Then, the server 5 sends an instruction to print a coloring picture corresponding to the selected answer candidates TB1, TB2 as an operation instruction corresponding to the selection information to the image processing device 2, and the image processing device 2 prints the coloring picture.

[0070] In this way, by displaying answer candidates TB1 and TB2 within the character information T, it is possible to present information to the user in an easy-to-understand manner, and it is also possible for the user to easily perform the operation of selecting answer candidates TB1 and TB2.

[0071] When the selection of answer candidates TB1, TB2 is performed by voice input to the voice device 3, information regarding the selection is transmitted to the server 5, and the server 5 transmits to the image processing device 2 an operational instruction to terminate the display of answer candidates TB1, TB2 and / or an invalidation instruction to invalidate the selection operation on the display unit 30. Then, based on the operational instruction and / or the invalidation instruction, the control unit 60 causes the display unit 30 to terminate the display of answer candidates TB1, TB2 and / or invalidates the selection operation on the display unit 30. This makes it possible to prevent duplicate selection operations from being performed.

[0072] According to the image processing device 2 of this embodiment described above, at least one of the text information corresponding to the voice notification by the audio device 3 and the two-dimensional code that can be converted into text information corresponding to the voice notification is displayed so that at least a portion of the display time of the text information and the two-dimensional code is included in the execution time of the voice notification. Therefore, even if the user has difficulty hearing the voice notification due to surrounding noise or misses it because they are concentrating on other work, the user can check the content of the voice notification on the display unit 30 of the image processing device 2, and can accurately understand it without missing it.

[0073] It should be noted that various modifications can be made to the image processing device 2 according to the above-described embodiment.

[0074] For example, the control unit 60 may cause the display unit 30 to hide text information corresponding to a previously made voice notification while the voice device 3 is making a voice notification. That is, before the next voice notification by the voice device 3 starts, the text information corresponding to the previously made voice notification is erased from the display of the display unit 30. This prevents old text information corresponding to old voice notifications from being displayed, making it possible to accurately understand the text information corresponding to the content of the currently made voice notification.

[0075] Furthermore, in the above-described embodiment, the control unit 60 causes the display unit 30 to display the text information T in a scrolling or page-switching manner, thereby making it possible to deal with cases where the entire text information T does not fit on the display unit 30. However, in addition to this mode, for example, the control unit 60 may cause the display unit 30 to display a portion of the text information and hide the other portions, and may cause the other portions to be displayed sequentially in response to audio notifications from the audio device 3.

[0076] Specifically, the control unit 60 may cause the display unit 30 to sequentially display the character information T from the beginning to the end as the voice notification by the voice device 3 progresses. At this time, the control unit 60 may cause the display unit 30 to switch the display range of the character information T from the beginning to the end. The control unit 60 may also cause the display unit 30 to switch the display of the character information T sentence by sentence from the beginning to the end. Alternatively, the control unit 60 may cause the display unit 30 to display the character information T character by character from the beginning to the end. These modes also make it possible to deal with cases where the entire character information T does not fit on the display unit 30.

[0077] FIG. 11 is a schematic diagram showing how a content image is displayed on the display unit 30 of the image processing device 2 according to the embodiment.

[0078] 11 , the display unit 30 of the image processing device 2 displays a content image 80 that is the target of image processing. In the example shown in Fig. 11 , the content image 80 is a map. The content image 80 is displayed on the display unit 30 based on an operation instruction given by voice via the audio device 3.

[0079] Specifically, when the audio device 3 receives an audio operation instruction, it transmits audio information including the operation instruction from the audio device 3 to the server 5, and the server 5 transmits a content image 80 based on the operation instruction to the image processing device 2, and the control unit 60 of the image processing device 2 displays the content image 80 on the display unit 30.

[0080] For example, an operation instruction such as "Please print a map of XX" is given via the audio device 3. The target area of ​​the map is specified by, for example, an address, an area name, a landmark facility (such as a station), etc. The audio device 3 issues audio notifications such as "Printing a map of XX" and "Is this map okay?", and the display unit 30 of the image processing device 2 displays text information corresponding to the audio notifications, such as "Printing a map of XX" and "Is this map okay?". Thereafter, the display unit 30 displays "Map of XX" as the content image 80. If the content image 80 is appropriate, the printing operation procedure proceeds.

[0081] FIG. 12 is a schematic diagram showing how a modified content image 81 obtained by modifying the magnification of a content image 80 is displayed on the display unit 30 of the image processing device 2 according to this embodiment.

[0082] 12 , the display unit 30 displays a changed content image 81 in which the magnification of the content image 80 has been changed from the first magnification to the second magnification, instead of the content image 80 shown in FIG. 11 . The display unit 30, for example, enlarges or reduces the map displayed as the content image 80 and displays the map with the changed magnification as the changed content image 81, based on the input operation received by the input unit 40. Specifically, the control unit 60 enlarges or reduces the content image 80 displayed by the display unit 30 based on the input operation received by the input unit 40. The display unit 30 displays operation buttons 90 (a magnification button (+ button) and a reduction button (− button)) as the input unit 40, and the content image 80 is changed to the changed content image 81 by performing an input operation using the operation buttons 90.

[0083] For example, if the user wants to change the content image 80 to an enlarged map, the user presses the operation button 90 of the input unit 40 (display unit 30) to display the enlarged map as the content image 81. If the content image 81 is appropriate, the user proceeds with the printing operation procedure.

[0084] As shown in FIG. 13, the display unit 30 may be configured to change the magnification from the magnification of the content image 80 (first magnification) to the magnification of the changed content image 81 (second magnification) by a pinch-out operation 91 or a pinch-in operation on the content image 80, and display the content image.

[0085] In addition, the display unit 30 may be configured to slide the changed content image 81 (or the content image 80) by a drag operation on the changed content image 81 (or the content image 80), as shown in Figure 14, and to display on the display unit 30 the part that was not displayed on the display unit 30 before the sliding.

[0086] Alternatively, the display unit 30 may be configured to change the display from the content image 80 to the changed content image 81 in response to an audio operation instruction via the audio device 3. For example, when the audio device 3 receives an audio operation instruction, the audio device 3 transmits audio information including the operation instruction to the server 5, the server 5 transmits the changed content image 81 based on the operation instruction, and the control unit 60 of the image processing device 2 causes the display unit 30 to display the changed content image 81.

[0087] For example, after the audio device 3 outputs a voice notification saying "Is this map OK?" and the image processing device 2 displays text information, the image processing device 2 displays the changed content image 81 (enlarged map) based on the voice instruction "Please change to an enlarged map" while displaying "Map of XX" as the content image 80.

[0088] 15, the display unit 30 may be configured to display an edge 92 of the printing paper or an edge 93 of the printable area of ​​the printing paper 92 as a print preview image together with the content image 80 or the changed content image 81. Note that the display unit 30 may be configured so that print settings such as the size, orientation, or margins of the printing paper can be specified by voice command.

[0089] 16 and 17, the operations of the image processing device 2 and the audio device 3 will be described using an example in which the content image is a coloring book.

[0090] As shown in Figures 16 and 17, when there are animal coloring books and vehicle coloring books available for coloring, the audio device 3 issues a voice notification saying, "Which would you like, the animal coloring book or the vehicle coloring book?", and the display unit 30 of the image processing device 2 displays text information corresponding to the voice notification saying, "Which would you like, the animal coloring book or the vehicle coloring book?"

[0091] When, for example, an animal coloring book is selected by an operation instruction given by voice or an operation instruction given by the input unit 40, an animal coloring book is displayed as a content image 82, as shown in Fig. 16. If the content image 82 is appropriate, the printing operation procedure is carried out.

[0092] For example, if the user wishes to change the content image 82 to a coloring book of a vehicle, the user can input buttons on the input unit 40 (display unit 30) or give a voice command to display a coloring book of a vehicle on the display unit 30 as a changed content image 83, in place of the content image 82, with the type of the content image 82 having been changed, as shown in Fig. 17. If the content image 83 is appropriate, the user proceeds with the printing procedure.

[0093] As described above, by displaying the content image that is the target of image processing based on an audio instruction on the display unit 30, the target image can be confirmed before the image processing is performed, thereby reducing unnecessary image processing. In the above-described embodiment, the image processing device 2 and the audio device 3 are separate devices, but they may also be integrated into an integrated device (audio-image processing device). That is, even in the audio-image processing device, the audio notification and text information or two-dimensional code are displayed on the display unit in synchronization with the audio notification. Even in this manner, even if the user has difficulty hearing the audio notification due to ambient noise or misses it while concentrating on other tasks, the user can still check the content of the audio notification on the display unit of the audio-image processing device, allowing them to accurately understand the notification without missing it.

[0094] (Modification) Although an example has been described in which the image processing device 2 has "a control unit 60 that causes the display unit 30 to display, in synchronization with the voice notification, character information T corresponding to the voice notification related to image processing by the voice device 3 and / or a two-dimensional code B that can be converted into character information T corresponding to the voice notification," such a control unit 60 may be provided in the server 5. In other words, the server 5 may be configured to remotely control the image processing device 2.

[0095] 18 is a functional block diagram of a server 5A according to the modified example. As shown in FIG. 18, the server 5A according to the modified example is configured to include a first communication unit 501, a second communication unit 502, and a control unit 503.

[0096] The first communication unit 501 performs data communication with an audio device 3 (see FIG. 1 ) having sound generation, sound collection, and communication functions. The communication may be performed directly with the audio device 3 or indirectly via a network N. Examples of the network include a telephone network, the Internet, a private network, and a LAN (Local Area Network). The communication may be wired or wireless.

[0097] The second communication unit 502 performs data communication with the image processing device 2 (see FIGS. 1 and 3 ), which is an image processing device 2 capable of operating in cooperation with the audio device 3 and has an image processing unit 20 that performs image processing and a display unit 30 that displays information related to the image processing. This communication may also be performed directly with the image processing device 2, or indirectly via the network N.

[0098] The control unit 503 controls the image processing device 2 via the second communication unit 502. Specifically, the control unit 503 causes the display unit 30 of the image processing device 2 to display, in synchronization with the voice notification, character information T corresponding to the voice notification regarding image processing by the voice device 3 and / or a two-dimensional code B that can be converted into character information corresponding to the voice notification.

[0099] The synchronization referred to here also includes (i) the display time of the text information and the execution time of the voice notification being the same, (ii) the display time of the text information and the execution time of the voice notification partially overlapping, and (iii) the display time of the text information and the execution time of the voice notification being different times but occurring consecutively.

[0100] Then, this server 5A performs the same operations as the control operations performed by the control unit 60 described in the above embodiment.

[0101] With this configuration, even if the user has difficulty hearing the voice notification due to surrounding noise or misses it because they are concentrating on other work, they can still check the content of the voice notification on the display unit 30 of the image processing device 2, so they can accurately understand the notification without missing it.

[0102] Other Embodiments The operational flows in the above-described embodiments do not necessarily have to be executed in chronological order according to the order depicted in the flow diagrams. For example, steps in the operations may be executed in an order different from that depicted in the flow diagrams, or may be executed in parallel. Furthermore, some steps in the operations may be deleted, or additional steps may be added to the process.

[0103] A program may be provided that causes a computer (image processing device) to execute the operations according to the above-described embodiments. The program may be recorded on a computer-readable recording medium. Using the computer-readable recording medium, the program can be installed on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory print medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a CD-ROM or a DVD-ROM.

[0104] As used in this disclosure, the terms "based on" and "depending on / in response to" do not mean "based only on" or "depending only on," unless expressly stated otherwise. The term "based on" means both "based only on" and "based at least in part on." Similarly, the term "depending on" means both "depending only on" and "depending at least in part on." Furthermore, the terms "include," "comprise," and variations thereof do not mean including only the listed items, but may mean including only the listed items or may include additional items in addition to the listed items. Furthermore, the term "or," as used in this disclosure, is not intended to mean an exclusive or. Furthermore, any reference to elements using designations such as "first," "second," etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some way. In this disclosure, where articles are added by translation, such as a, an, and the in English, these articles shall include the plural unless the context clearly indicates otherwise.

[0105] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention.

[0106] (Additional Notes) Additional notes will be given regarding the features of the above-described embodiment.

[0107] Supplementary Note 1: An image processing device capable of receiving operation instructions by voice via an audio device having a voice notification function, comprising: an image processing unit that performs image processing; and a display unit that displays information related to the image processing, wherein the display unit displays a content image that is the target of the image processing based on the operation instruction by voice.

[0108] Supplementary Note 2: The image processing device according to Supplementary Note 1, wherein the display unit displays, in place of the content image, a changed content image obtained by changing the magnification of the content image from a first magnification to a second magnification.

[0109] Supplementary Note 3: The image processing device according to Supplementary Note 2, wherein the display unit displays the changed content image based on an operation instruction by voice.

[0110] Supplementary Note 4: The image processing device according to Supplementary Note 1, wherein the display unit displays a changed content image in which the type of the content image has been changed, in place of the content image.

[0111] Supplementary Note 5: The image processing device according to Supplementary Note 4, wherein the display unit displays the changed content image based on an operation instruction by voice.

[0112] Supplementary Note 6: The image processing device according to Supplementary Note 1, wherein the display unit displays text information corresponding to the voice notification by the audio device and / or an information code convertible into text information corresponding to the voice notification, in synchronization with the voice notification.

[0113] Supplementary Note 7: The image processing device according to Supplementary Note 6, wherein the display unit displays the character information and / or the information code so that at least a part of a display time of the character information and / or the information code is included in an execution time of the voice notification.

[0114] Supplementary Note 8: The image processing device according to Supplementary Note 6, wherein the display unit, while the audio device is performing the audio notification, does not display text information and / or the information code corresponding to an audio notification performed before the audio notification.

[0115] Supplementary Note 9: The image processing device according to Supplementary Note 6, wherein when a portion of the character information is displayed and another portion is not displayed, the display unit is configured to enable confirmation of the other portion of the character information by scrolling or switching pages.

[0116] Supplementary Note 10: The image processing device according to Supplementary Note 6, wherein the display unit displays at least a portion of the character information in a user-selectable state, and when a portion of the character information is selected, displays related information related to the selected character information.

[0117] Supplementary Note 11: The image processing device according to Supplementary Note 10, wherein the selectable character information is at least one of a word, a sentence, and a paragraph included in the character information.

[0118] Supplementary Note 12: The image processing device according to Supplementary Note 10, wherein the selectable character information is displayed in a manner different from that of character information that is not selectable character information.

[0119] Supplementary Note 13: The image processing device according to Supplementary Note 10, further comprising an input unit that accepts an input operation from a user, wherein the input unit accepts an operation from the user to select the selectable character information.

[0120] Supplementary Note 14: The image processing device according to Supplementary Note 10, wherein the related information is information that embodies the content of the selectable character information or character information that is different from the selectable character information but is associated with the selectable character information.

[0121] Supplementary Note 15: The image processing device according to Supplementary Note 6, wherein the display unit sequentially displays the character information from the beginning to the end as the audio notification by the audio device progresses.

[0122] Supplementary Note 16: The image processing device according to Supplementary Note 15, wherein the sequential display of the character information is performed by switching a display range from the beginning to the end of the character information, by switching the display on a sentence-by-sentence basis, or by displaying each character one by one.

[0123] Supplementary Note 17 The image processing device according to Supplementary Note 6, further comprising an input unit that accepts an input operation from a user, wherein the display unit displays answer candidates of the user in response to the voice notification, and the input unit accepts an operation from the user to select at least one of the answer candidates.

[0124] Supplementary Note 18: The image processing device according to Supplementary Note 17, wherein the display unit displays a plurality of objects indicating the answer candidates in response to the audio device outputting a question to the user by the audio notification.

[0125] Supplementary Note 19: When a selection of the answer candidate displayed on the display unit is made by voice input to the voice device, the display unit terminates the display of the answer candidate, and / or the input unit invalidates the answer input by the user. The image processing device described in Supplementary Note 17.

[0126] Supplementary Note 20: The image processing device according to any one of Supplementary Notes 1 to 19, wherein the text information corresponding to the voice notification is a text version of the voice included in the voice notification.

[0127] 1: Image processing system 2: Image processing device 3: Audio device 4: Access point 5, 5A: Server 30: Display unit 40: Input unit 40b: Physical button 50: Communication unit 60: Control unit 70: Storage unit 501: First communication unit 502: Second communication unit 503: Control unit

Claims

1. An image processing apparatus capable of receiving an operation instruction by voice via a voice device having a voice notification function, the apparatus comprising: an image processing unit that performs image processing; and a display unit that displays information related to the image processing, wherein the display unit displays a content image that is an object of the image processing based on the operation instruction by voice.

2. The image processing apparatus according to claim 1, wherein the display unit displays a changed content image whose magnification is changed from a first magnification to a second magnification of the content image instead of the content image.

3. The image processing apparatus according to claim 2, wherein the display unit displays the changed content image based on an operation instruction by voice.

4. The image processing apparatus according to claim 1, wherein the display unit displays a changed content image whose type is changed instead of the content image.

5. The image processing apparatus according to claim 4, wherein the display unit displays the changed content image based on an operation instruction by voice.

6. The image processing apparatus according to claim 1, wherein the display unit displays character information corresponding to the voice notification by the voice device and / or an information code convertible into the character information corresponding to the voice notification in synchronization with the voice notification.

7. The image processing apparatus according to claim 6, wherein the display unit displays the character information and / or the information code such that at least a part of the display time of the character information and / or the information code is included in the execution time of the voice notification.

8. The image processing apparatus according to claim 6, wherein during the execution of the voice notification by the voice device, the display unit makes the character information and / or the information code corresponding to a voice notification performed before the voice notification non-displayed.

9. The image processing apparatus according to claim 6, wherein when a part of the character information is displayed and the other part is non-displayed, the other part of the character information can be confirmed by scroll display or page switching.

10. The image processing apparatus according to claim 6, wherein the display unit displays at least a part of the character information in a selectable state by a user, and when a part of the character information is selected, the display unit displays related information related to the selected character information.

11. The image processing apparatus according to claim 10, wherein the selectable character information is at least one of words, sentences, and paragraphs included in the character information.

12. The image processing apparatus according to claim 10, wherein the selectable character information is displayed in a manner different from that of character information that is not the selectable character information.

13. The image processing apparatus according to claim 10, further comprising an input unit that receives a user input operation, wherein the input unit receives an operation for selecting the selectable character information from the user.

14. The image processing apparatus according to claim 10, wherein the related information is information that concretizes the content of the selectable character information or character information that is different from the selectable character information but is associated with the selectable character information.

15. The image processing apparatus according to claim 6, wherein the display unit sequentially displays from the beginning to the end of the character information as the voice notification by the voice device progresses.

16. The image processing apparatus according to claim 15, wherein sequentially displaying the character information is performed by switching the display range from the beginning to the end of the character information, switching the display by sentence unit, or displaying one character at a time.

17. The image processing apparatus according to claim 6, further comprising an input unit that receives a user input operation, wherein the display unit displays answer candidates for the voice notification, and the input unit receives an operation for selecting at least one of the answer candidates from the user.

18. The image processing apparatus according to claim 17, wherein the display unit displays a plurality of objects indicating the answer candidates in response to the voice device outputting a question to the user by the voice notification.

19. The image processing apparatus according to claim 17, when the selection for the answer candidates displayed on the display unit is made by voice input to the voice device, the display unit ends the display of the answer candidates, and / or the input unit invalidates the user's answer input.

20. The image processing apparatus according to any one of claims 1 to 19, wherein the character information corresponding to the voice notification is the result of converting the voice included in the voice notification into characters.

Images