Information processing method, information processing apparatus, and image forming apparatus

Abstract

The present application provides an information processing method, an information processing device, and an image forming device, which can change a display method of external content. The information processing method includes displaying external content generated by an external system based on specific data acquired by the information processing device on an operation screen. The information processing method includes, in a case where a specific operation on the external content is detected by the operation device, causing the external system to generate external content whose display method is changed according to the specific operation, and displaying the changed external content on the operation screen.

Description

Technical Field

[0001] This disclosure relates to an information processing method, an information processing apparatus, and an image forming apparatus for displaying an operation screen on a display device. Background Technology

[0002] As a related technology, an information processing device that displays an operation screen provided by a web server is known to the public. In this information processing device, when a "preview" button displayed on the operation screen is operated, a preview image is generated based on image data based on file information selected in a file selection screen, and the generated preview image is displayed on the operation screen.

[0003] In the structure shown in the aforementioned related technologies, the information processing device can also send image data to a server and receive a preview image (external content) generated by the server (external system) based on the image data, thereby displaying the preview image on the operation screen.

[0004] However, in the structure shown in the above-mentioned related technologies, since the external content is generated by an external system, there is a problem in the information processing device, for example, that the display method of the external content cannot be changed. Summary of the Invention

[0005] The purpose of this invention is to provide an information processing method, an information processing apparatus, and an image forming apparatus, which can change the display mode of external content generated by an external system and displayed on the operation screen of the information processing apparatus.

[0006] One aspect of the present invention provides an information processing method for controlling the display of the display device within an information processing apparatus that includes a display device for displaying an operation screen and an operation device for detecting operations on the operation screen. The information processing method includes displaying external content generated by an external system based on specific data obtained by the information processing apparatus on the operation screen. The information processing method further includes performing a change process when the operation device detects a specific operation on the external content. The change process involves causing the external system to generate the external content with a changed display mode according to the specific operation, and displaying the changed external content on the operation screen.

[0007] Other aspects of the information processing apparatus of the present invention include one or more processors that perform the above-described information processing method, the display device, and the operating device.

[0008] Other aspects of the image forming apparatus of the present invention include an information processing device and an image forming unit that forms an image on a thin sheet. The operating device is capable of accepting operations related to the image forming unit from the operating screen.

[0009] According to the present invention, an information processing method, an information processing apparatus, and an image forming apparatus can be provided, and the display mode of external content generated by an external system and displayed on the operation screen of the information processing apparatus can be changed.

[0010] This specification is introduced by way of summarizing the concepts set forth in the following detailed description, with appropriate reference to the accompanying drawings. This specification is not intended to limit the essential and fundamental features of the subject matter recited in the claims, nor is it intended to limit the scope of the subject matter recited in the claims. Furthermore, the object recited in the claims is not limited to embodiments that address some or all of the shortcomings recounted in any part of the invention. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating the structure of the image forming apparatus according to an embodiment.

[0012] Figure 2 This is a diagram showing an example of an operation screen displayed in an image forming apparatus according to an embodiment.

[0013] Figure 3 This is a diagram illustrating an example of an operation screen displaying external content in an image forming apparatus according to an embodiment.

[0014] Figure 4 This is a sequence diagram illustrating an example of the display processing of external content in an image forming apparatus according to an embodiment.

[0015] Figure 5 This is a sequence diagram illustrating an example of external content modification processing in an image forming apparatus according to an embodiment.

[0016] Figure 6 This is a diagram illustrating a first example of displaying external content in an image forming apparatus according to an embodiment.

[0017] Figure 7 This is another example of a second display example of external content in an image forming apparatus according to an embodiment.

[0018] Figure 8 This is a diagram illustrating a third example of the display of external content in an image forming apparatus according to an embodiment.

[0019] Figure 9 This is a diagram illustrating a fourth example of displaying external content in an image forming apparatus according to an embodiment.

[0020] Figure 10 This is a fifth display example diagram illustrating the external content in the image forming apparatus of the embodiment. Detailed Implementation

[0021] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, the following embodiments are merely examples embodying the present invention and are not intended to limit the scope of the invention.

[0022] (1) Overall structure of the image forming apparatus First, refer to Figure 1 The overall structure of the image forming apparatus 10 in this embodiment will be described. For example... Figure 1 As shown, the image forming apparatus 10 includes a printing device 1 that performs printing processing and a user interface device 3. Here, printing processing is the process of forming an image on a sheet body 91. The image forming apparatus 10 is, for example, a printer, copier, fax machine, or digital multifunction printer.

[0023] The user interface device 3 includes a touch panel 3a for detecting touch operations and a display device 3b capable of displaying images. The touch panel 3a is disposed on the screen of the display device 3b. The touch panel 3a is an example of an operation device for detecting operations. Hereinafter, the touch panel 3a will sometimes be referred to as "operation device 3a". The image forming apparatus 10 is an example of an information processing apparatus equipped with an operation device and a display device 3b.

[0024] exist Figure 1 In the example shown, the image forming apparatus 10 includes a printing device 1, an image reading device 2, a user interface device 3, a communication device 4, a control device 5, and an auxiliary storage device 6. The image forming apparatus 10 is capable of communicating with an external system 8 via a network 80. The network 80 includes, for example, a LAN (Local Area Network) or the Internet.

[0025] External system 8 is, for example, an information processing device such as a server, personal computer, tablet computer, or smartphone. External system 8 may also have functions that the image forming apparatus 10, as an information processing device, does not possess. In this embodiment, as described later, external system 8 has the function of generating chart images, a function that the image forming apparatus 10 does not have. Furthermore, external system 8 may, for example, have the same functions as the image forming apparatus 10, and its processing power when using these functions is higher than that of the image forming apparatus 10 when using these functions.

[0026] The printing apparatus 1 performs printing processing in a predetermined manner, such as electrophotography or inkjet printing. The printing apparatus 1 includes a paper feeding mechanism (not shown) for transporting a sheet 91, and an image forming unit (not shown) for forming an image on the transported sheet 91. The printing apparatus 1 performs printing processing based on data from a read image obtained by the image reading device 2 or by receiving print data. Here, the received print data is data included in a print request received from an external system 8 or other device via the communication device 4.

[0027] Image reading device 2 performs image reading processing. Here, image reading processing is the process of reading an image from the original document 92. Image reading device 2 includes a light scanning mechanism (not shown) that scans light onto the original document 92, and an image sensor (not shown) that receives reflected light reflected by the original document 92. The image sensor outputs data of the read image. Here, the read image is an image read from the original document 92.

[0028] Communication device 4 is a communication interface device that enables communication between the network 80 and other devices such as external system 8. Control device 5 transmits and receives all data with the aforementioned other devices through communication device 4. Communication between communication device 4 and other devices can be wired or wireless.

[0029] Furthermore, the communication device 4 is capable of performing image transmission processing. Here, image transmission processing refers to the process of sending data of the read image obtained by the image reading device 2 to a recipient address specified by the user via the network 80. In this embodiment, image transmission processing includes network fax transmission processing for sending the read image data via the network 80.

[0030] The control device 5 performs various calculations, data processing, and controls the various electrical devices included in the image forming apparatus 10. Furthermore, the control device 5 controls the display of the display device 3b by controlling the display device 3b.

[0031] The control device 5 includes a CPU (Central Processing Unit) 51 and RAM (Random Access Memory) 52, etc. The CPU 51 is one or more processors that execute computer programs to perform various data processing and control functions. In addition, one or more other processors such as DSP (Digital Signal Processor) can also replace the CPU 51 to perform the above-mentioned data processing and control functions.

[0032] RAM52 is a computer-readable volatile storage device. RAM52 temporarily stores computer programs executed by CPU51, as well as data output and referenced during the execution of computer programs by CPU51.

[0033] Auxiliary storage device 6 is a computer-readable, non-volatile storage device. Auxiliary storage device 6 can store computer programs and various data. Examples of auxiliary storage device 6 include SSDs (Solid State Drives) and other flash memory or hard disk drives.

[0034] The control device 5 is capable of performing data recording processing. Here, data recording processing is the process of recording the data of the read image obtained by the image reading device 2 or the received print data obtained by the communication device 4 into the auxiliary storage device 6.

[0035] CPU 51 has multiple processing modules implemented by executing the computer program described above. In this embodiment, CPU 51 has multiple processing modules such as an operation control unit 5a, a UI (User Interface) control unit 5b, and a device management unit 5c.

[0036] The job control unit 5a performs job control. Here, job control means causing each device of the image forming apparatus 10 to perform image processing corresponding to the request input via the touch panel 3a or the communication device 4. The job control unit 5a controls the printing device 1, the image reading device 2, and the communication device 4, etc. The job control unit 5a causes some or all of the printing device 1, the image reading device 2, and the communication device 4 to perform the requested image processing. The image processing mentioned here includes printing processing performed by the printing device 1, image reading processing performed by the image reading device 2, image transmission processing performed by the communication device 4, and data recording processing performed by the control device 5, etc.

[0037] The UI control unit 5b is capable of performing display control processing. Here, display control processing refers to the process of causing the display device 3b to display various information related to the image forming apparatus 10. In this embodiment, the UI control unit 5b causes the display device 3b to display the operation screen P1 (see reference 10). Figure 2 The operation screen P1 is used to input various settings of the image forming apparatus 10 and instructions for operation control to be performed by the image forming apparatus 10. The operation screen P1 is a user interface installed with native code. In other words, the operation screen P1 is implemented by a native application installed when manufacturing the image forming apparatus 10. The UI control unit 5b causes the display device 3b to display the operation screen P1 containing multiple operation icons I1, and causes the display device 3b to display a screen corresponding to the operation of selecting any one of the multiple operation icons I1. In this embodiment, the operation of selecting any one of the multiple operation icons I1 is detected by the touch panel 3a, which is an operation device.

[0038] Figure 2This is an example of an operation screen P1 displayed on display device 3b. For example... Figure 2 As shown, the upper part of the operation screen P1 displays an operation icon I1, "Copy," indicating that the printing device 1 performs printing processing, and an operation icon I1, "Send," indicating that the communication device 4 performs image transmission processing. Furthermore, the upper part of the operation screen P1 displays an operation icon I1, "Status," indicating the display of the job performed by the image forming apparatus 10, and an operation icon I1, "Device Information," indicating the status display of the image forming apparatus 10. Additionally, the upper part of the operation screen P1 displays an operation icon I1, "Language," indicating the language of the string displayed on the operation screen P1.

[0039] On the left side of the operation screen P1, there are operation icons I1 indicating various settings of the image forming apparatus 10, namely "Device Settings," and operation icons I1 indicating errors generated in the image forming apparatus 10, namely "Notification Reports." Furthermore, on a sub-screen of the operation screen P1, there are operation icons I1 indicating various function settings of the image forming apparatus 10, namely "Function Settings," and operation icons I1 indicating various settings of the network 80 to which the image forming apparatus 10 is connected, namely "Network Settings." Moreover, on the left side of the operation screen P1, there are operation icons I1 indicating various settings related to the security of the image forming apparatus 10, namely "Security Settings."

[0040] On the right side of the operation screen P1, there are operation icons I1 indicating the size of the sheet 91, namely "paper size", and operation icons I1 indicating the multi-page combination process of combining multiple originals onto the sheet 91, namely "multi-page combination". In addition, on the right side of the operation screen P1, there are operation icons I1 indicating whether to print on one or both sides of the sheet 91, namely "single-sided / double-sided", and operation icons I1 indicating whether to print in black and white or in color on the sheet 91, namely "black and white / color".

[0041] Furthermore, the operation screen P1 displays an instruction as external content C1 (see below). Figure 3 The operation icon I1 that displays the chart is called "chart display". Hereinafter, the operation icon I1 that indicates the display of external content C1 will also be referred to as "operation icon I11". In addition, in this embodiment, operation icon I11 is "chart display", but of course it will vary depending on how the external content C1 is displayed.

[0042] The device management unit 5c determines various states of the image forming apparatus 10 and controls the operation of the image forming apparatus 10 based on the determination results. Furthermore, the device management unit 5c exports and records data indicating the operating status of the image forming apparatus 10, including the printing unit 1. Here, the data indicating the operating status includes, for example, power consumption data, consumable consumption data, or greenhouse gas emission data.

[0043] For example, the device management unit 5c determines whether a sleep state is established. When a sleep state is established, it switches the image forming apparatus 10 from the normal mode to a power-saving mode that consumes less power than the normal mode. Here, the sleep state may include, for example, a state in which the power switch of the image forming apparatus 10 is operated in a specific way or the state in which the image forming apparatus 10 is not operated for a specified period of time or more.

[0044] Furthermore, based on the detection results of the power detection circuit (not shown) provided by the image forming apparatus 10, the device management unit 5c derives the hourly, daily, and monthly power consumption of the image forming apparatus 10, and records the power consumption data representing the derived power consumption results in the auxiliary storage device 6.

[0045] Furthermore, the device management unit 5c, based on the detection results of various sensors (not shown) provided by the image forming apparatus 10, derives the hourly, daily, and monthly consumption amounts of consumables, and records the consumable consumption data, representing the derived consumption amounts, in the auxiliary storage device 6. Here, the consumption amounts of consumables include, for example, the amount of developer consumed during the printing process or the number of sheets 91 used.

[0046] Furthermore, the equipment management unit 5c converts the exported electricity and consumable consumption into greenhouse gas emissions, and records the hourly, daily, and monthly greenhouse gas emissions data in the auxiliary storage device 6. Greenhouse gases include, for example, carbon dioxide.

[0047] However, as a related technology, an information processing device that displays an operation screen provided by a web server is already known to the public. In this information processing device, when the "preview" button displayed on the operation screen is operated, a preview image is generated based on image data based on file information selected in the file selection screen, and the generated preview image is displayed on the operation screen.

[0048] In the structure shown in the above-mentioned related technologies, the information processing device can also send image data to a server and receive a preview image (external content) generated by the server (external system) based on the image data, thereby displaying the preview image on the operation screen.

[0049] However, in the structure shown in the above-mentioned related technologies, since the external content is generated by an external system, there is a problem in the information processing device, for example, that the display method of the external content cannot be changed.

[0050] In contrast, in this embodiment, the processing described below, performed by the information processing device, enables the information processing method, the information processing device, and the image forming apparatus 10 to change the display mode of external content.

[0051] Specifically, the information processing method of this embodiment is as follows: The display of the display device 3b in the information processing apparatus (image forming apparatus 10) is controlled. The information processing apparatus includes the display device 3b for displaying an operation screen P1 and an operation device 3a for detecting operations in the operation screen P1. Furthermore, the CPU 51 (one or more processors) of the information processing apparatus (image forming apparatus 10) performs communication processing, display processing, and change processing.

[0052] In other words, the information processing apparatus includes: a display device 3b for displaying the operation screen P1; an operation device 3a for detecting operations in the operation screen P1; and one or more processors (CPU 51) that implement the above-described information processing method. Furthermore, the image forming apparatus 10 includes the above-described information processing apparatus and a printing apparatus 1 (image forming unit) for forming an image on the sheet body 91. Moreover, the operation screen P1 can receive operations related to the printing apparatus 1 (image forming unit) in the operation screen P1.

[0053] The communication process is as follows: specific data acquired by the information processing device (image forming device 10) is sent to the external system 8, and external content C1 generated by the external system 8 based on the specific data is received. Specifically, the CPU 51 controls the communication device 4, thereby sending specific data to the external system 8 via the network 80, and receiving external content C1 from the external system 8 via the network 80.

[0054] Here, the specific data refers to data indicating the operating status of the image forming apparatus 10, such as power consumption data, consumable consumption data, or greenhouse gas emission data, stored in the auxiliary storage device 6 of the image forming apparatus 10. Furthermore, the specific data is not limited to data pre-stored in the auxiliary storage device 6 of the image forming apparatus 10; for example, it could be data acquired by the image forming apparatus 10 in real time. In this case, the specific data is not stored in the auxiliary storage device 6 but is sent to the external system 8. Additionally, the specific data is not limited to data indicating the operating status of the image forming apparatus 10; any data held by the image forming apparatus 10 is acceptable.

[0055] When the external system 8 receives specific data sent from the information processing device (image forming apparatus 10), it generates external content C1 based on the received specific data. Here, the external content C1 is not generated by a native application installed on the image forming apparatus 10, but by the external system 8. The external content C1 is, for example, general web content generated using programming languages ​​such as HTML (Hypertext Markup Language), CSS (Cascading Style Sheets), or JavaScript.

[0056] In this embodiment, one example of external content C1 is an image containing a bar chart generated based on power consumption data from the image forming apparatus 10. In other words, external content C1 is a chart image C12 generated based on specific data. Furthermore, external content C1 is a chart image generated based on data from a specific period within the specific data. The specific period is expressed in months or years, for example. Additionally, external content C1 is not limited to an image containing a bar chart, as long as it can be displayed on the display device 3b.

[0057] In this embodiment, the image forming apparatus 10, which is an information processing device, does not have the specific function of generating an image containing a bar graph based on power consumption data and displaying it on the display device 3b. In other words, the information processing device (image forming apparatus 10) does not have the specific function of generating specific content based on specific data. Therefore, the information processing device sends the specific data (power consumption data) stored in its own device to the external system 8. Then, the information processing device uses the specific function of the external system 8 to generate external content C1 (an image containing a bar graph based on power consumption data) instead of the image forming apparatus 10. That is, the external system 8 has a specific function and generates specific content as external content C1 based on specific data. As a result, it has the advantage that content using functions not possessed by the information processing device can be displayed on the display device 3b without updating the native application of the information processing device.

[0058] The display processing involves displaying the received external content C1 on the operation screen P1 while it is superimposed on the operation screen P1. In other words, the display processing involves displaying the external content C1 generated by the external system 8 based on specific data obtained by the information processing apparatus (image forming apparatus 10) on the operation screen P1. Specifically, the CPU 51 (especially the UI control unit 5b) displays the operation screen P1 on the display device 3b using a native application pre-installed on the information processing apparatus (image forming apparatus 10). Then, when the CPU 51 receives the external content C1 from the external system 8 via the network 80, it displays the received external content C1 on the operation screen P1 while it is superimposed on the operation screen P1, in other words, while it is embedded in the operation screen P1. In this embodiment, the CPU 51 displays the external content C1 on the operation screen P1 using the function of displaying web content through a web browser. That is, the CPU 51 causes the display device 3b to display both the operation screen P1 generated by the native application and the external content C1 displayed by the web browser.

[0059] Figure 3 This is a diagram illustrating an example of an operation screen P1 containing external content C1. Figure 2 In the operation screen P1 shown, after the user operates on the "Chart Display" operation icon I11, the following is displayed: Figure 3 The operation screen P1 is shown. Figure 3 In the example shown, the outer content C1 includes multiple operation icons C11 (hereinafter referred to as the first operation icon C11a to the fifth operation icon C11e) and a chart image C12 representing a bar chart generated based on power consumption data. Figure 3 In the example shown, graph image C12 displays a bar graph representing the daily power consumption of the image forming apparatus 10 in September 2024.

[0060] exist Figure 3 In the example shown, a first operation icon C11a indicating the year of the displayed object and a second operation icon C11b indicating the month of the displayed object are displayed at the top of the external content C1. Furthermore, a third operation icon C11c is also displayed at the top of the external content C1, indicating the type of data representing the operating status of the displayed object. A fourth operation icon C11d is displayed at the lower right of the external content C1, indicating either the display of a chart image C12 representing daily power consumption or a chart image C12 representing monthly power consumption. A fifth operation icon C11e is displayed at the upper right of the external content C1, indicating that the display of the external content C1 has stopped.

[0061] exist Figure 3In the example shown, external content C1 is displayed to the right of the column of vertically arranged operation icons I1, such as "Device Settings," on the operation screen P1. Furthermore, while external content C1 is displayed, multiple operation icons C11a to C11e, such as "Paper Size," located in positions overlapping with the display area of ​​external content C1, are not displayed. Alternatively, external content C1 can be displayed on the operation screen P1 while the multiple operation icons C11a to C11e, such as "Paper Size," are not displayed, but rather overlapped with them.

[0062] In the change processing, when the operating device 3a detects a specific operation on the external content C1, the external system 8 generates external content C1 with a changed display mode according to the specific operation, and displays the changed external content C1 on the operation screen P1. Specifically, when the operating device 3a detects a specific operation on the external content C1, the CPU 51 (especially the UI control unit 5b) sends data indicating a change in the display mode of the external content C1 corresponding to the specific operation to the external system 8. Upon receiving the data, the external system 8 generates external content C1 with a changed display mode based on the received data, and sends the generated changed external content C1 to the information processing device (image forming apparatus 10). Then, when the information processing device receives the changed external content C1, it displays the received changed external content C1 on the operation screen P1.

[0063] Here, the specific operation includes the operation of a user touching the display area of ​​external content C1 displayed on the operation screen P1 with their finger. In particular, in this embodiment, the specific operation includes the operation of a user touching the display area of ​​the chart image C12 in the external content C1 displayed on the operation screen P1 with their finger. Regarding the display example of the modified external content C1 corresponding to the specific operation, a detailed description will be given in "[3] Display Example of External Content" described later.

[0064] Furthermore, the display mode mentioned here is not the display mode of the information processing device (image forming apparatus 10), but rather the display mode of the external content C1. Moreover, "changing the display mode of the external content C1" means changing the method of displaying specific data within the external content C1 without changing the content of the specific data that forms the basis for generating the external content C1. For example, as described later, when a chart image C12 based on specific data within a specific period is the external content C1, the method of displaying the specific data in the chart image C12 is changed by changing the specific period. In this case, changing the specific period of the chart image C12 is equivalent to changing the display mode of the external content C1.

[0065] [2] Example of operation of information processing device (image forming device) The following is for reference Figure 4 and Figure 5 An example of the processing sequence performed by the information processing apparatus (image forming apparatus 10) and an example of the information processing method of this embodiment will be described. Here, steps S101, S102, ... denote the numbers of the processing sequence (steps) performed by the information processing apparatus or the external system 8.

[0066] First, refer to Figure 4 Here is an example of the display processing of external content C1 performed by the information processing device. This processing begins, for example, when the user instructs the display of external content C1 by operating the operation icon I11 (here, the operation icon for "chart display") on the operation screen P1.

[0067] (Step S101) First, CPU 51 generates a native region by executing a native application. Here, the native region is a display area on display device 3b that includes multiple operation icons I1, etc., displayed by the native application; it is a display area outside the display area of ​​external content C1 in operation screen P1. The generation of the native region is achieved, for example, by parsing XML content stored in auxiliary storage device 6.

[0068] (Step S102) Next, CPU 51 reads specific data from auxiliary storage device 6. Then, CPU 51 controls communication device 4 to send the read specific data and an instruction signal instructing external system 8 to generate external content C1 via network 80. This instruction signal can be implemented, for example, via an HTTP (Hypertext Transfer Protocol) request. Alternatively, CPU 51 can also send specific data acquired in real time to external system 8 via network 80 instead of reading the specific data from auxiliary storage device 6.

[0069] (Step S103) When the external system 8 receives an instruction signal from the information processing device (image forming device 10) via the network 80, it generates external content C1 based on the specific data contained in the instruction signal.

[0070] (Step S104) Secondly, the external system 8 sends a response signal containing the generated external content C1 to the information processing device (image forming device 10) via the network 80. The aforementioned response signal is implemented, for example, via an HTTP response.

[0071] (Step S105) CPU 51 controls communication device 4 to generate a display area for external content C1 contained in the response signal when it receives a response signal from external system 8 via network 80. The generation of the display area for external content C1 is achieved, for example, by performing syntax analysis on the external content C1 using a native application.

[0072] (Step S106) Then, CPU 51 displays the generated local area and the display area of ​​external content C1 on display device 3b. Thus, operation screen P1 containing external content C1 is displayed on display device 3b.

[0073] Secondly, refer to Figure 5 This describes an example of how an information processing device processes changes to external content C1. This processing can be performed while the external content C1 is displayed on the operation screen P1.

[0074] (Step S201) First, the operating device 3a detects a specific operation on the external content C1. Specifically, the operating device 3a detects a specific operation when the user performs it in the display area of ​​the external content C1.

[0075] (Step S202) Next, based on the specific operation detected by the operating device 3a, the CPU 51 generates a change indication signal that includes a change indication of the display mode of the external content C1. Then, the CPU 51 controls the communication device 4 to send the generated change indication signal to the external system 8 via the network 80. The aforementioned change indication signal is implemented, for example, via an HTTP request.

[0076] (Step S203) When the external system 8 receives a change instruction signal from the information processing apparatus (image forming apparatus 10) via the network 80, it generates external content C1 based on the instruction contained in the change instruction signal. Here, the external system 8 does not generate new external content C1, but rather generates external content C1 with a changed display mode of the already generated external content C1. Therefore, the time required to generate the changed external content C1 in the external system 8 can be shorter than in the case of generating new external content C1.

[0077] (Step S204) Secondly, the external system 8 sends a response signal containing the generated modified external content C1 to the information processing device (image forming device 10) via network 80. The aforementioned response signal is implemented, for example, via an HTTP response.

[0078] (Step S205) CPU 51 controls communication device 4 to receive a response signal sent from external system 8 via network 80. Then, CPU 51 controls display device 3b to display the modified external content C1 contained in the response signal on the display area of ​​external content C1. Thus, operation screen P1 containing the modified external content C1 is displayed on display device 3b.

[0079] In addition, Figure 5 In the example shown, since the display area of ​​the external content C1 remains unchanged, the CPU 51 does not regenerate the display area, but displays the modified external content C1 on the already generated display area. However, this is not a limitation. For example, if a specific operation includes an operation that indicates a change in the display position of the display area of ​​the external content C1, such as enlarging or shrinking it, the CPU 51 may also regenerate the display area of ​​the external content C1 based on that operation.

[0080] Hereafter, whenever the user performs a specific operation, steps S201 to S205 described above are executed. As a result, the external content C1 displayed on the operation screen P1 is updated in real time according to the user's operation.

[0081] As described above, in the information processing apparatus (image forming apparatus 10) of this embodiment, external content C1 generated by the external system 8 based on specific data possessed by the information processing apparatus (image forming apparatus 10) can be displayed on the operation screen P1. Therefore, in the information processing apparatus of this embodiment, new functions or new information that are difficult to achieve through the native application can be added to the operation screen P1 as external content C1. Therefore, in the information processing apparatus of this embodiment, since the operation screen P1 can be customized through external content C1 even without updating the native application, it has the advantage of easily customizing the operation screen P1 of the target device.

[0082] Furthermore, in the information processing apparatus (image forming apparatus 10) of this embodiment, during the display of external content C1 on the operation screen P1, the external content C1 whose display mode has been changed according to a specific operation on the external content C1 can be displayed on the operation screen P1. Therefore, in the information processing apparatus of this embodiment, the display mode of the external content C1 can be changed. Thus, in the information processing apparatus of this embodiment, the display mode of the external content C1 can be flexibly changed according to the user's operation.

[0083] Furthermore, in the information processing apparatus (image forming apparatus 10) of this embodiment, since the display mode of the external content C1 is changed in real time according to the user's specific operation on the external content C1, it has the advantage that the user can easily and intuitively operate the external content C1. Moreover, for example, if the display mode of the external content C1 is changed to emphasize the information that the user wants to obtain from the external content C1, it has the advantage that the user can easily grasp that information in a relatively short time.

[0084] [3] Example of displaying external content The following is for reference Figures 6 to 10 The display example of external content C1 in the operation screen P1 is explained. Figures 6 to 10 The illustrations only show the display area of ​​external content C1, omitting the display areas outside the display area of ​​external content C1 in the operation screen P1. Furthermore, the first to fifth display examples shown below are examples of display examples of external content C1 in the operation screen P1, and the display examples of external content C1 are not limited to these examples.

[0085] (First example shown) In the first display example, when the specific operation is the first operation, the CPU 51 displays a chart image C12 that has been modified for a specific period on the operation screen P1. The first operation is the operation of modifying the specific period in the chart image C12, which is external content C1. Therefore, it has the advantage that the user can freely modify the specific period, and that the chart image C12 corresponding to the period the user wants to view in the specific data can be displayed on the operation screen P1.

[0086] Figure 6 This indicates the first display example of external content C1 in the operation screen P1. For example... Figure 6 As shown in the <Before Change> section, before the user performs a specific operation, a chart image C12 representing the power consumption of the image forming apparatus 10 in September 2024 is displayed as external content C1 on the operation screen P1.

[0087] Here, for example, the user performs a drawing operation (first operation) in chart image C12, drawing a straight line from left to right. Thus, as... Figure 6 As shown in the <Modified> section, the CPU 51 displays a chart image C12 representing the power consumption of the image forming apparatus 10 in October 2024 as external content C1 on the operation screen P1. That is, in the first display example, whenever the user performs a first operation, a chart image C12 changing the specific period to one month later can be displayed on the operation screen P1.

[0088] Additionally, for example, if a user performs a stroke operation (first operation) in the chart image C12 to draw a straight line from right to left, the chart image C12 with the specific period changed to one month ago can also be displayed on the operation screen P1.

[0089] Furthermore, for example, if a user performs a zoom-out operation (first operation) on the chart image C12 to depict the trajectory between two fingers, the chart image C12, which increases the unit size for a specific period, can also be displayed on the operation screen P1. Specifically, in Figure 6 In the chart image C12 shown in the <before the change>, when the user performs a zoom-out operation, the chart image C12 representing the annual power consumption of the image forming apparatus 10 is displayed on the operation screen P1.

[0090] Furthermore, for example, if the user performs a zoom-in operation (first operation) on the chart image C12, depicting the trajectory between two fingers, the chart image C12 can be zoomed out to a smaller unit for a specific period, which can also be displayed on the operation screen P1. Specifically, in the chart image C12 representing the annual power consumption of the image forming apparatus 10, if the user performs a zoom-in operation, the chart image C12 representing the monthly power consumption of the image forming apparatus 10 will be displayed on the operation screen P1.

[0091] Furthermore, the first operation described above involves drawing a specific trajectory in the chart image C12, but it is not limited to this. For example, the first operation could also be a touch operation that selects any position in the chart image C12.

[0092] (Second example) In the second display example, when the specific operation is the second operation, the CPU 51 displays the chart image C12 on the operation screen P1, highlighting one region of the two regions of the chart image C12 divided based on the first value selected by the second operation, compared to the other region. The second operation is the operation of selecting the value of any axis in the chart image C12 as the first value. This has the advantage that the user can freely change the first value, and that the chart image C12 highlighting the data that the user particularly wants to view from specific data can be displayed on the operation screen P1.

[0093] Figure 7 This represents a second display example of external content C1 within the operation screen P1. For example... Figure 7 As shown in the <Before Change> section, before the user performs a specific operation, a chart image C12 representing the power consumption of the image forming apparatus 10 in September 2024 is displayed as external content C1 on the operation screen P1.

[0094] Here, for example, the user performs a touch operation (second operation) in chart image C12 to select "450W / h" (first value) for the Y-axis. Thus, as... Figure 7 As shown in the <Modified> section, CPU 51 displays a chart image C12 on the operation screen P1 that emphasizes the area above "450W / h" on the Y-axis (one side's area) compared to the area below "450W / h" on the Y-axis (the other side's area), as external content C1. For example, CPU 51 sets the display color of each bar in the area above "450W / h" on the Y-axis to a brighter color (e.g., red) than the display color of each bar in the area below "450W / h" on the Y-axis. Furthermore, CPU 51 may also apply a shading line to each bar in the area above "450W / h" on the Y-axis.

[0095] Furthermore, for example, if a user performs a touch operation (second operation) in the chart image C12 to select any value (first value) on the X-axis, the chart image C12 that emphasizes the area above the first value on the X-axis compared to the area below the first value on the X-axis is displayed on the operation screen P1.

[0096] In addition, Figure 7 In the example shown, a chart image C12 that emphasizes areas above the first value compared to areas below the first value is displayed on the operation screen P1, but this is not a limitation. For example, a chart image C12 that emphasizes areas below the first value compared to areas above the first value could also be displayed on the operation screen P1. Switching the emphasized areas could be performed, for example, if an action such as detecting two consecutive touches of the first value by the user is detected.

[0097] <Third Display Example> In the third display example, when the specific operation is the third operation, the CPU 51 displays the chart image C12, normalized based on the second value selected through the third operation, on the operation screen P1. The third operation is the operation of selecting the value of any axis in the chart image C12 as the second value. Therefore, when the user compares multiple chart images C12, since each chart image C12 is normalized using the second value, it has the advantage of facilitating the comparison of each chart image C12.

[0098] Figure 8 This represents the third display example of external content C1 in the operation screen P1. For example... Figure 8 As shown in the <Chart Image of September 2024 Before Change>, before the user performs a specific operation, a chart image C12 representing the power consumption of the image forming apparatus 10 in September 2024 is displayed as external content C1 on the operation screen P1.

[0099] Here, for example, the user performs a sliding operation (third operation) in chart image C12, drawing a circle around the Y-axis indicating "600W / h" (second value). The CPU 51 then normalizes chart image C12 by displaying the selected "600W / h" as the maximum value on the Y-axis. Figure 8 In the <Chart Image of September 2024 Before Change>, since "600W / h" was already the maximum value displayed on the Y-axis before the third operation was performed, the display method of chart image C12 remained unchanged. On the other hand, in Figure 8 In the revised chart image for October 2024, the display method of chart image C12 has changed. Specifically, in the original chart image C12 for October 2024, the maximum value displayed on the Y-axis was "300W / h", while in the revised chart image C12 for October 2024, the maximum value displayed on the Y-axis is "600W / h".

[0100] Additionally, for example, if the user performs a dragging operation (third operation) in the chart image C12 where a circle surrounds any value on the X-axis (second value), the CPU 51 can also display the selected second value as the maximum value on the X-axis and normalize the chart image C12. Specifically, for example, if the maximum value on the X-axis is "31st", in months where there is no day corresponding to "31st", the CPU 51 can also dim the display of that day compared to other days, or not display that day at all.

[0101] <Fourth Display Example> In the fourth display example, when the specific operation is the fourth operation, the CPU 51 displays a chart image C12 on the operation screen P1, which includes the determination result of whether the data corresponding to the axis selected by the fourth operation exceeds the third value selected by the fourth operation. Here, a notification image C13 indicating that the data has exceeded the third value is displayed on the operation screen P1. The fourth operation is the operation of selecting the value of any axis in the chart image C12 as the third value. Therefore, since the notification image C13 indicates that the data has exceeded the threshold (third value), it has the advantage that the user can easily grasp that the data has exceeded the threshold in a relatively short time.

[0102] Figure 9 This represents the fourth display example of external content C1 in the operation screen P1. For example... Figure 9 As shown in the <Before Change> section, before the user performs a specific operation, a chart image C12 representing the power consumption of the image forming apparatus 10 in September 2024 is displayed on the display device 3b as external content C1.

[0103] Here, for example, a user touches the Y-axis value "540W / h" (the third value) with two fingers in chart image C12, and then performs a stroke operation (the fourth operation) to draw a straight line from left to right with two fingers. Thus, as... Figure 9 As shown in the <Modified> section, the CPU 51, for example, emphasizes "540W / h" on the Y-axis and displays it on the operation screen P1 via a dialog box image C14. Subsequently, if the CPU 51 finds data in a specific dataset that exceeds "540W / h" on the Y-axis (here, the power consumption of the image forming apparatus 10), it displays a notification image C13 on the operation screen P1, for example, a dialog box image indicating a message that "the target has been exceeded."

[0104] However, for example, when the user instructs the printing process of the sheet 91 to be performed on the operation screen P1, the data (here, the power consumption of the image forming apparatus 10) may sometimes exceed the third value. In this case, the CPU 51 may also display a notification image C13 on the operation screen P1.

[0105] Additionally, the third value selected through the fourth operation (here, "540W / h" on the Y-axis) can also be displayed without emphasis on the operation screen P1.

[0106] In addition, the fourth operation may be, for example, the user selecting a setting icon pre-set in the external content C1 and entering a third value in the setting screen displayed on the display device 3b by selecting the setting icon.

[0107] <Fifth Display Example> In the fifth display example, when the specific operation is the fifth operation, the CPU 51 displays external content C1, whose display mode has been changed according to the parameters set by the fifth operation, on the operation screen P1. The fifth operation is the operation of setting parameters related to the display mode of the operation screen P1. For example, this parameter is a threshold value for the brightness of the operation screen P1. Therefore, for example, when the parameter is a threshold value for the brightness of the operation screen P1, since the display mode of the external content C1 is changed according to the brightness of the operation screen P1, it has the advantage of improving the visual recognizability of the external content C1.

[0108] Figure 10 This represents the fifth display example of external content C1 in the operation screen P1. For example, the following operation (the fifth operation) is performed: the user selects a setting icon preset in the external content C1, and inputs parameters (here, the brightness threshold of the operation screen P1) in the setting screen displayed on the display device 3b by selecting the setting icon. Then, the CPU 51 changes the display color of each bar in the chart image C12 according to the set brightness threshold of the operation screen P1. Specifically, as follows... Figure 10As shown in <Brightness: Small>, when the brightness of the operation screen P1 is less than the above threshold and the operation screen P1 is relatively dark, the CPU 51 changes the display color of each bar in the chart image C12 to a brighter color. Furthermore, as... Figure 10 As shown in <Brightness: Large>, when the brightness threshold of the operation screen P1 is greater than the above threshold and the operation screen P1 is relatively bright, the CPU51 changes the display color of each bar in the chart image C12 to a darker color.

[0109] Furthermore, the parameter that can be set via the fifth operation is not limited to the brightness of the operation screen P1. For example, if the image forming apparatus 10 is equipped with an illuminance sensor, this parameter can also be an illuminance threshold. In this case, if the illuminance detected by the illuminance sensor is less than the aforementioned threshold, the CPU 51 changes the display color of each bar in the chart image C12 to a brighter color. Conversely, if the illuminance detected by the illuminance sensor is greater than the aforementioned threshold, the CPU 51 changes the display color of each bar in the chart image C12 to a darker color.

[0110] (Variation example) In the above embodiments, the specific operations (first operation to fifth operation) represented by the first to fifth display examples of external content C1 are only examples, and the operation mode is not limited to these specific operations. That is, each specific operation in the first to fifth display examples can be any operation mode as long as the operation mode can be distinguished from the specific operations in other display examples.

[0111] In the above embodiment, the external content C1 is content generated by the external system 8 using a function not possessed by the target device (image forming apparatus 10), but it is not limited to this. For example, the external content C1 may also be content generated by the external system 8 using a function possessed by the target device. Specifically, although the target device has a specific function to generate an image containing bar graphs using power consumption data, it may sometimes lack the processing power to perform that specific function. In this case, if the external system 8, which has that specific function and higher processing power than the target device, performs that specific function to generate the image, the image can be displayed on the operation screen P1 more quickly compared to the case where the image is generated by the target device.

[0112] Furthermore, in the above embodiments, external content C1 may also include, for example, content that can only be displayed on a web browser and is not generated based on specific data.

[0113] (Notes on the invention) The following is a summary of the invention extracted from the above embodiments. Furthermore, the various components and processing functions described in the following notes can be arbitrarily combined by selection.

[0114] (Note 1) An information processing method controls the display of the display device in an information processing apparatus that includes a display device for displaying an operation screen and an operation device for detecting operations in the operation screen. The information processing method includes: External content generated by an external system based on specific data obtained by the information processing device is displayed on the operation screen; and When the operating device detects a specific operation on the external content, a change process is performed. In the change process, the external system generates the external content whose display mode has been changed according to the specific operation, and displays the changed external content on the operating screen.

[0115] (Note 2) In the information processing methods described in Appendix 1 The external content is a chart or image generated based on data from a specific period within the specific data set. In the change processing, if the specific operation is a first operation that changes the specific period in the chart image, the chart image with the changed specific period is displayed on the operation screen.

[0116] (Note 3) In the information processing methods described in Appendix 1 or 2 The external content refers to charts and images generated based on the specific data. In the change processing, if the specific operation is a second operation that selects the value of any axis in the chart image as a first value, the chart image that is highlighted by comparing one of the two regions of the chart image divided based on the first value with the other region is displayed on the operation screen.

[0117] (Note 4) In any of the information processing methods in notes 1 to 3, The external content refers to charts and images generated based on the specific data. In the change processing, if the specific operation is a third operation that selects the value of any axis in the chart image as the second value, the chart image normalized based on the second value is displayed on the operation screen.

[0118] (Note 5) In any of the information processing methods in notes 1 to 4, The external content refers to charts and images generated based on the specific data. In the change processing, if the specific operation is a fourth operation that selects the value of any axis in the chart image as the third value, the chart image including the determination result of whether the data corresponding to the axis in the specific data exceeds the third value will be displayed on the operation screen.

[0119] (Note 6) In any of the information processing methods in notes 1 to 5, In the change processing, if the specific operation is the fifth operation that sets parameters related to the display mode of the operation screen, the external content whose display mode has been changed according to the parameters will be displayed on the operation screen.

[0120] (Note 7) In any of the information processing methods in notes 1 to 6, The information processing device does not have the specific function of generating specific content based on the specific data. The external system has the specific function and generates the specific content as the external content based on the specific data.

[0121] (Note 8) An information processing apparatus, comprising: One or more processors that execute any one of the information processing methods in notes 1 to 7; The display device; and The operating device.

[0122] (Note 9) An image forming apparatus, comprising: The information processing device described in Appendix 8; and An image forming unit that forms an image on a thin sheet. The operating device can accept operations related to the image forming unit in the operating screen.

[0123] The scope of this invention is not limited to the foregoing description, but is defined by the claims. Therefore, the embodiments described in this specification are merely illustrative and not intended to limit the scope. Thus, all modifications that do not depart from the scope and limits of the claims, as well as content equivalent to the scope and limits of the claims, are included within the scope of the claims.

Claims

1. An information processing method, characterized in that, The information processing method controls the display of the display device in the information processing device, which includes a display device for displaying an operation screen and an operation device for detecting operations in the operation screen. The information processing method includes: External content generated by an external system based on specific data obtained by the information processing device is displayed on the operation screen; and When the operating device detects a specific operation on the external content, a change process is performed. In the change process, the external system generates the external content whose display mode has been changed according to the specific operation, and displays the changed external content on the operating screen.

2. The information processing method according to claim 1, characterized in that, The external content is a chart or image generated based on data from a specific period within the specific data set. In the change processing, if the specific operation is a first operation that changes the specific period in the chart image, the chart image with the changed specific period is displayed on the operation screen.

3. The information processing method according to claim 1, characterized in that, The external content refers to charts and images generated based on the specific data. In the change processing, if the specific operation is a second operation that selects the value of any axis in the chart image as a first value, the chart image that is highlighted by comparing one of the two regions of the chart image divided based on the first value with the other region is displayed on the operation screen.

4. The information processing method according to claim 1, characterized in that, The external content refers to charts and images generated based on the specific data. In the change processing, if the specific operation is a third operation that selects the value of any axis in the chart image as the second value, the chart image normalized based on the second value is displayed on the operation screen.

5. The information processing method according to claim 1, characterized in that, The external content refers to charts and images generated based on the specific data. In the change processing, if the specific operation is a fourth operation that selects the value of any axis in the chart image as the third value, the chart image including the determination result of whether the data corresponding to the axis in the specific data exceeds the third value will be displayed on the operation screen.

6. The information processing method according to any one of claims 1 to 5, characterized in that, In the change processing, if the specific operation is the fifth operation that sets parameters related to the display mode of the operation screen, the external content whose display mode has been changed according to the parameters will be displayed on the operation screen.

7. The information processing method according to any one of claims 1 to 5, characterized in that, The information processing device does not have the specific function of generating specific content based on the specific data. The external system has the specific function and generates the specific content as the external content based on the specific data.

8. An information processing device, characterized in that, include: One or more processors that perform the information processing method according to any one of claims 1 to 5; The display device; as well as The operating device.

9. An image forming apparatus, characterized in that, include: The information processing apparatus according to claim 8; as well as An image forming unit that forms an image on a thin sheet. The operating device can accept operations related to the image forming unit in the operating screen.

Images