Image forming apparatus, terminal apparatus, and operating method

Abstract

The invention provides an image forming apparatus, a terminal apparatus, and an operation method, which can improve user convenience when a part or all of an operation image of the image forming apparatus is displayed from the terminal apparatus via a network and an operation is performed. An image forming apparatus according to one embodiment of the present invention includes a communication unit, a display unit that displays an operation image, a transmission processing unit, and a reception processing unit. The transmission processing unit transmits display information obtained on the basis of the operation image displayed on the display unit to the terminal device via the communication unit. The reception processing unit receives the operation event received by the terminal device from the terminal device via the communication unit. When first information including a plurality of predetermined operation events is received by the reception processing unit, the transmission processing unit transmits, as display information, first display information indicating a response result to the plurality of predetermined operation events indicated by the first information to the terminal device.

Description

Technical Field

[0001] The present invention relates to an image forming apparatus, a terminal device, and an operation method. Background Art

[0002] In Patent Document 1, a remote control panel for a function of displaying an operation unit of an image forming apparatus via a network is disclosed, and a system for stably and highly speedily performing animation transmission in the remote control panel without depending on the line speed of a public line and the geographical conditions of the line is disclosed.

[0003] However, in the remote control panel, there is a case where it takes time to reflect a user operation event, resulting in a reduction in convenience. In addition, the technology described in Patent Document 1 is not a technology that can solve the above problems.

[0004] Therefore, there is a need to develop a technology that shortens the time required to reflect a user operation event when displaying and operating a part or all of an operation image of an image forming apparatus from a terminal device via a network, thereby improving user convenience.

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2019-220075 Summary of the Invention

[0006] An image forming apparatus according to an aspect of the present invention includes: a communication unit; a display unit that displays an operation image; a transmission processing unit that transmits display information obtained based on the operation image displayed on the display unit to a terminal device via the communication unit; and a reception processing unit that receives an operation event received by the terminal device from the terminal device via the communication unit. When the reception processing unit receives first information including a predetermined plurality of operation events, the transmission processing unit transmits first display information indicating a response result to the predetermined plurality of operation events shown in the first information to the terminal device as the display information.

[0007] A terminal device according to one aspect of the present invention includes: an operation unit; a communication unit; a reception processing unit that receives display information obtained based on an operation image displayed on a display unit of an image forming apparatus via the communication unit from the image forming apparatus; a display unit that displays the operation image based on the display information received by the reception processing unit; a transmission processing unit that transmits an operation event received by the operation unit to the image forming apparatus via the communication unit, the transmission processing unit transmits first information including a predetermined plurality of operation events received by the operation unit to the image forming apparatus, and when the first information is transmitted by the transmission processing unit, the reception processing unit receives, as the display information, first display information indicating a response result to the predetermined plurality of operation events shown in the first information from the image forming apparatus.

[0008] In an operation method according to one aspect of the present invention, a transmission process is executed, the transmission process being a process in which an image forming apparatus including a communication unit and a display unit that displays an operation image transmits display information obtained based on the operation image displayed on the display unit to a terminal device via the communication unit, a reception process is executed, the reception process being a process in which the image forming apparatus receives an operation event received by the terminal device from the terminal device via the communication unit, and in the transmission process, when first information including a predetermined plurality of operation events is received by the reception process, first display information indicating a response result to the predetermined plurality of operation events shown in the first information is transmitted to the terminal device as the display information via the communication unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A block diagram showing a structural example of an image forming system including an image forming apparatus and a terminal device according to an embodiment.

[0010] Figure 2 For explaining Figure 1 A flowchart for explaining a processing example of a terminal device in the image forming system.

[0011] Figure 3 For explaining Figure 1 A flowchart for explaining a processing example when a mouse down operation occurs in a terminal device in the image forming system.

[0012] Figure 4 For explaining Figure 1Flowchart for explaining the processing example when a mouse up operation occurs in a terminal device in an image forming system.

[0013] Figure 5 For explaining Figure 1 Flowchart for explaining the processing example when a mouse click operation is received by an image forming device in an image forming system.

[0014] Figure 6 For explaining Figure 1 Flowchart for explaining the processing example during a mouse click operation in an image forming system.

[0015] Figure 7 To represent Figure 6 Schematic diagram showing the transition example of the display image in the processing example.

[0016] Figure 8 Flowchart for explaining the processing during a mouse click operation in the image forming system related to the comparative example.

[0017] Figure 9 For explaining Figure 1 Flowchart for explaining the processing example during an icon movement operation in an image forming system.

[0018] Figure 10 To represent Figure 9 Schematic diagram showing the transition example of the display image in the processing example.

[0019] Figure 11 To represent Figure 9 Of the display image in the processing example, subsequent to Figure 10 Transition example schematic diagram.

[0020] Figure 12 Flowchart for explaining the processing during an icon movement operation in the image forming system related to the comparative example.

[0021] Figure 13 To represent Figure 12 Schematic diagram showing the transition example of the display image in the processing.

[0022] Figure 14 Schematic block diagram showing another structural example of an image forming system including the image forming device and the terminal device according to the embodiment.

[0023] Figure 15 Diagram showing an example of the hardware structure of the device. Detailed implementation mode

[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, each drawing is merely an illustration for explaining the embodiments of the present invention. Furthermore, not all of the structural elements described in the embodiments of the present invention are necessarily essential structural elements of the present invention.

[0025] Use Figure 1 to describe an image forming system including an image forming apparatus and a terminal apparatus according to an embodiment. Figure 1 It is a block diagram showing a structural example of the image forming system.

[0026] As Figure 1 shown, an image forming system (hereinafter referred to as the present system) 100 according to the present embodiment can include an image forming apparatus 1, a terminal apparatus 2, a reception server apparatus 3, and a base server apparatus 4. The terminal apparatus 2 can be connected to the reception server apparatus 3 via a network N. In addition, the reception server apparatus 3 can be connected to the base server apparatus 4 via the network N. In addition, the base server apparatus 4 can be connected to the image forming apparatus 1 via the network N. Hereinafter, a case where the present system 100 adopts such a network structure example will be described.

[0027] In addition, although the same network N is used to represent the network connecting between each device for simplicity of description in Figure 1 , different networks can be used to connect some or all of the devices, and the connection method of each device is not limited thereto. In addition, the present system 100 can also adopt a network structure other than the above network structure example. For example, as long as the present system 100 includes an image forming apparatus 1 and a terminal apparatus 2 connected to the image forming apparatus via a network N, it can be configured not to include a reception server apparatus 3 and a base server apparatus 4. In addition, the present system 100 can be a system in which the functions of the reception server apparatus 3 and the base server apparatus 4 are mounted on the same server apparatus, or can be a system including three or more distributed server apparatuses having further functions. That is, the image forming apparatus 1 only needs to be configured to communicate with the terminal apparatus 2 via at least one information processing apparatus such as at least one server apparatus. In this case, the terminal apparatus 2 is configured to communicate with the image forming apparatus 1 via at least one information processing apparatus such as at least one server apparatus.

[0028] This system is a system that can operate the operation image of the image forming apparatus 1 from the terminal device 2. Thus, the operation image provided from the image forming apparatus 1 to the terminal device 2, or the provision of the operation image can also be referred to as a remote control panel or the like. The acceptance server device 3 can be set as a server device that provides acceptance processing of operations from the terminal device 2 for the terminal device 2. The basic server device 4 can be set as a server device that provides processing for operating the operation image of the image forming apparatus 1 based on the operations accepted by the acceptance server device 3 to the image forming apparatus 1.

[0029] Outline structure of the image forming apparatus 1

[0030] As Figure 1 shown, the image forming apparatus 1 can include a control unit 10, a storage unit 11, a display unit 12, an operation unit 13, a communication unit 14, and a printing unit 15.

[0031] The control unit 10 of the image forming apparatus 1 can also be referred to as a controller and implements overall control of the image forming apparatus 1. Although the detailed content of the control unit 10 will be described later, it can include a transmission processing unit 10a and a reception processing unit 10b.

[0032] The control unit 10 can be configured, for example, to include an arithmetic processing device such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a working memory, and a storage device that stores control programs and parameters. The control unit 10 can also be configured as a SoC (System on a Chip). From these examples, it can also be seen that the control unit 10 can be set to store the control program in an executable state. However, the control unit 10 can also be set to store the control program as a circuit structure like an FPGA (field-programmable gate array), or can be configured as a dedicated circuit. The transmission processing unit 10a and the reception processing unit 10b can be installed as the above programs. The above programs can include programs for implementing the functions of the transmission processing unit 10a and the reception processing unit 10b while working in cooperation with the storage unit 11, the display unit 12, the operation unit 13, and the communication unit 14. Or, the above programs can include programs for implementing the functions of the transmission processing unit 10a and the reception processing unit 10b while working in cooperation with the storage unit 11, the display unit 12, the operation unit 13, the communication unit 14, and the printing unit 15.

[0033] The storage unit 11 is, for example, a hard disk drive, a solid state drive, or other memory. A part of the memory included in the control unit 10 may also be regarded as the storage unit 11. The storage unit 11 may also be regarded as a part of the control unit 10.

[0034] The display unit 12 is a part for displaying information. As the information, the display unit 12 can display an operation image as a user interface (UI) image. This operation image can be pre-stored in the storage unit 11. The display unit 12 is constituted by, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 12 can also be configured to include a display and a drive circuit for driving the display.

[0035] The operation unit 13 is a part for receiving operations and inputs performed by the user, and thus can also be called an operation reception unit. The operation unit 13 can include a touch panel or a touch sensor mounted on the display unit 12. In this case, by the user performing a touch operation on the operation image displayed on the display unit 12, the operation unit 13 can receive an operation corresponding to the area where the touch operation was performed. The display unit 12 that displays the operation image and the operation unit 13 having a touch panel or a touch sensor mounted on the display unit 12 can constitute an operation panel of the image forming apparatus 1. In addition, the operation unit 13 may, for example, also include a physical button group. A part of the physical button group can also be configured to be able to receive operations on the operation image. Of course, a part or all of the physical button group can be configured to receive operations independent of the operation image.

[0036] The communication unit 14 can be one or more communication interfaces for enabling the image forming apparatus 1 to perform communication with a plurality of external devices in a wired or wireless manner according to a predetermined communication protocol including a predetermined communication standard. The external device is, for example, a device having a communication function such as a personal computer (PC), a server device, a smart phone, or a tablet type terminal. In the above network structure example, in the case of performing communication related to processing related to the operation image, the external device includes the base server device 4.

[0037] The communication unit 14 can send display information for causing the terminal device 2 to display the operation image to the terminal device 2 via the base server device 4 and the reception server device 3. Additionally, the display information may also be the operation image itself. Furthermore, the communication unit 14 can receive an operation event received by the terminal device 2 from the terminal device 2 via the reception server device 3 and the base server device 4.

[0038] In addition, the communication unit 14 can receive a print target file from an external device. The external device that sends the print target file can be set as a terminal device other than the terminal device 2. However, the communication unit 14 can also be configured to receive the print target file directly from the terminal device 2 or via the acceptance server device 3 and the base server device 4.

[0039] The printing unit 15 executes printing on the medium based on, for example, a print target file received from a terminal device other than the terminal device 2 or from the terminal device 2 via the communication unit 14, or a print target file obtained by image reading performed by a scanner (not shown). Thus, an image represented by the print target file is formed on the medium. The printing method in the printing unit 15 is not limited, and various printing methods such as an inkjet printing method and a laser printing method can be applied. When the image forming apparatus 1 adopts the inkjet printing method, it can be called an inkjet printer, and when it adopts the laser printing method, it can be called a laser printer.

[0040] Outline Structure of Terminal Device 2

[0041] The terminal device 2 can be set as a device with a communication function such as a smart phone, a tablet terminal, or a PC. The terminal device 2 is not limited to a portable type and can also be a stationary type device. As Figure 1 illustrated, the terminal device 2 can include a control unit 20, a storage unit 21, a display unit 22, an operation unit 23, and a communication unit 24.

[0042] The control unit 20 of the terminal device 2 can also be called a controller and implements overall control of the terminal device 2. Although the detailed content of the control unit 20 will be described later, it can include a reception processing unit 20a and a transmission processing unit 20b.

[0043] The control unit 20 can be configured, for example, to include an arithmetic processing device such as a CPU and a GPU, a working memory, and a storage device that stores control programs and parameters. The control unit 20 can also be configured as an SoC. From these examples, it can also be seen that the control unit 20 can be configured to store control programs in an executable state. However, the control unit 20 can also be configured to store control programs as a circuit structure like an FPGA, or can be configured as a dedicated circuit. The reception processing unit 20a and the transmission processing unit 20b can be installed as the above programs. The above programs can be set as programs for implementing the functions of the reception processing unit 20a and the transmission processing unit 20b while working in cooperation with the storage unit 21, the display unit 22, the operation unit 23, and the communication unit 24.

[0044] The storage unit 21 is, for example, a hard disk drive, a solid state drive, or other memory. A part of the memory included in the control unit 20 can also be regarded as the storage unit 21. The storage unit 21 can also be regarded as a part of the control unit 20.

[0045] The display unit 22 is a part for displaying information, and is constituted by, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 22 can also be configured to include a display and a drive circuit for driving the display.

[0046] The operation unit 23 is a part that receives operations and inputs performed by the user, and can also be called an operation reception unit. The operation unit 23 can be realized by, for example, any one or more of a physical button, a touch panel or a touch sensor mounted on the display unit 12, a pointing device, and a keyboard. In a configuration where the operation unit 23 includes a touch panel or a touch sensor, the display unit 12 and the touch panel are included, and it can also be called the operation panel of the terminal device 2.

[0047] The communication unit 24 can be one or more communication interfaces for the terminal device 2 to perform communication with one or more external devices in a wired or wireless manner according to a predetermined communication protocol including a predetermined communication standard. In the above network structure example, in the case of performing communication related to processing related to the operation image, the external device will include the reception server device 3.

[0048] The communication unit 24 can receive display information for causing the terminal device 2 to display an operation image from the image forming device 1 via the reception server device 3 and the base server device 4. As described above, the display information can also be the operation image itself. In addition, the communication unit 24 can send the operation event received by the operation unit 23 to the image forming device 1 via the base server device 4 and the reception server device 3.

[0049] In addition, the communication unit 24 can also be configured to be able to directly send a print target file to the image forming device 1 or to the image forming device 1 via the reception server device 3 and the base server device 4.

[0050] Outline structure of the reception server device 3 and the base server device 4

[0051] Although the details of the reception server device 3 and the base server device 4 are omitted, the reception server device 3 and the base server device 4 can each be a server computer having functions of receiving operations and managing at least a remote control panel. Although not shown, any server computer can include a control unit, a communication unit, and a storage unit.

[0052] The control unit can be configured, for example, to include arithmetic processing devices such as a CPU and a GPU, a working memory, and a storage device that stores control programs, parameters, and the like. The control unit can also be configured as an SoC. From these examples, it can also be seen that the control unit can be configured to store the control program in an executable state. However, the control unit can also be configured to store the control program as a circuit structure like an FPGA, or can be configured as a dedicated circuit.

[0053] The control unit of the reception server device 3 receives display information from the image forming device 1 via the base server device 4 and sends the display information to the terminal device 2. In addition, the control unit of the reception server device 3 receives an operation event from the terminal device 2 and sends the operation event to the image forming device 1 via the base server device 4. The control unit of the reception server device 3 can install functions related to such information transmission and reception as a program, for example.

[0054] The control unit of the base server device 4 receives display information from the image forming device 1 and sends the display information to the terminal device 2 via the reception server device 3. In addition, the control unit of the base server device 4 receives an operation event from the terminal device 2 via the reception server device 3 and sends the operation event to the image forming device 1. The control unit of the base server device 4 can install functions related to the transmission and reception of such information as a program, for example.

[0055] The above communication unit can be set as one or more communication interfaces for a server computer to perform wired or wireless communication with one or more external devices according to a predetermined communication protocol including a predetermined communication standard. In the above network structure example, the communication unit in the reception server device 3 can communicate with the terminal device 2 and the base server device 4 as external devices. In the above network structure example, the communication unit in the base server device 4 can communicate with the image forming device 1 and the reception server device as external devices.

[0056] The above storage unit is, for example, a hard disk drive, a solid state drive, or other memory. A part of the memory included in the above control unit can also be regarded as this storage unit. In other words, the storage unit in the server computer can also be regarded as a part of the control unit in the server computer.

[0057] In the above example of the network structure, the storage unit in the acceptance server device 3 stores information for managing communication with the basic server device 4, information for managing communication with a specific terminal device 2, and the like. By managing communication with the specific terminal device 2, access from a terminal device that is not stored, that is, an unregistered terminal device, to the acceptance server device 3 can be restricted.

[0058] In the above example of the network structure, the storage unit in the basic server device 4 stores information for managing communication with the acceptance server device 3, device information of a specific image forming device 1, and information for managing communication with the image forming device 1, and the like. By managing communication with the specific image forming device 1, access from an image forming device that is not stored, that is, an unregistered image forming device, to the acceptance server device 3 can be restricted.

[0059] The specific structure of the present system 100

[0060] Details of the control unit 10 of the image forming device 1 will be described together with details of the terminal device 2. In the present system 100, as a schematic process, the terminal device 2 accesses the acceptance server device 3 to receive display information obtained based on an operation image displayed on the display unit 12 of the image forming device 1, and based on the display information, causes the display unit 22 to display the operation image. Moreover, a user operation on the operation image is accepted by the operation unit 23 of the terminal device 2, and an operation event indicating the user operation is transmitted to the acceptance server device 3. The acceptance server device 3 transmits the operation event to the image forming device 1 via the basic server device 4. The image forming device 1 accepts the operation of the operation image based on the received operation event, and thus the operation image can be changed by a transition to the next operation image or the like.

[0061] First, an example of the structure on the image forming device 1 side will be described. As described above, the control unit 10 of the image forming device 1 can include a transmission processing unit 10a and a reception processing unit 10b.

[0062] The transmission processing unit 10a transmits the display information obtained based on the operation image displayed on the display unit 12 to the terminal device 2 via the communication unit 14. In the above network structure example, the transmission of such display information by the transmission processing unit 10a is executed via the basic server device 4 and the reception server device 3. Since the display information is the information to be displayed on the screen of the display unit 12, it can also be called screen information. In addition, the display information can be displayed, for example, using a web browser in the display unit 22 of the terminal device 2. In this case, the display information can also be called web screen information. As shown in this example, the transmission processing unit 10a may also transmit the display information as the information for the browser of the terminal device 2 to display.

[0063] The reception processing unit 10b receives the operation event received by the terminal device 2 from the terminal device 2 via the communication unit 14. In the above network structure example, the reception of such operation events by the reception processing unit 10b is executed via the reception server device 3 and the basic server device 4. The operation event is an event indicating an operation received via the operation image displayed based on the display information transmitted to the terminal device 2. In addition, since the reception processing unit 10b obtains the operation event from the terminal device 2, it can also be called an acquisition processing unit.

[0064] The image forming apparatus 1 has the function of remotely providing an operation panel to the terminal device 2, that is, the function of providing a remote control panel, by such transmission and reception of information realized by the transmission processing unit 10a and the reception processing unit 10b.

[0065] The transmission processing unit 10a in the present embodiment further performs the following processing. That is, when the reception processing unit 10b receives the first information including a predetermined plurality of operation events, the transmission processing unit 10a transmits, as the display information, the first display information indicating the response result to the predetermined plurality of operation events indicated by the first information to the terminal device 2.

[0066] As described for the operation event, the reception of the first information is also executed via the communication unit 14. In the above network structure example, the reception of such first information by the reception processing unit 10b is also executed via the reception server device 3 and the basic server device 4. In addition, the first display information is transmitted as the display information when the above conditions are satisfied. Therefore, the transmission of the first information realized by the transmission processing unit 10a is also executed via the communication unit 14. In the above network structure example, the transmission of such first display information by the transmission processing unit 10a is also executed via the basic server device 4 and the reception server device 3.

[0067] Next, a structural example on the side of the terminal device 2 will be described. As described above, the control unit 20 of the terminal device 2 can include a reception processing unit 20a and a transmission processing unit 20b.

[0068] The reception processing unit 20a receives display information obtained based on the operation image displayed on the display unit 12 of the image forming device 1 from the image forming device 1 via the communication unit 24. In the above-described network structure example, the reception of such display information by the reception processing unit 20a is executed via the base server device 4 and the acceptance server device 3. In addition, since the reception processing unit 20a acquires display information from the image forming device 1, it can also be referred to as an acquisition processing unit.

[0069] Moreover, the display unit 22 displays the operation image based on the display information received by the reception processing unit 20a. In addition, as described above, in the display unit 22, the display information can be displayed using, for example, a Web browser or the like. As shown in this example, the reception processing unit 20a may also receive the display information as information for displaying the browser of the terminal device 2.

[0070] In addition, the transmission processing unit 20b transmits the operation event received by the operation unit 23 to the image forming device 1 via the communication unit 24. In the above-described network structure example, the transmission of such an operation event by the transmission processing unit 20b is executed via the acceptance server device 3 and the base server device 4. As such an operation event, for example, an event of a click operation or a touch operation on the operation image displayed on the display unit 22 implemented by an indicating device can be cited. In addition, hereinafter, it will be described in such a way that the indicating device is called a mouse and the operation implemented by the mouse is called a mouse operation.

[0071] The terminal device 2 has a function of remotely enjoying the provision of the operation panel from the image forming device 1 through the transmission and reception of such information implemented by the reception processing unit 20a and the transmission processing unit 20b, that is, it has a function of enjoying a remote control panel.

[0072] The transmission processing unit 20b and the reception processing unit 20a in the present embodiment further perform the following processing. That is, the transmission processing unit 20b transmits first information including a predetermined plurality of operation events received by the operation unit 23 to the image forming device 1. As described for the operation event, the transmission of the first information is also executed via the communication unit 24. In the above-described network structure example, the transmission of such first information by the transmission processing unit 20b is also executed via the acceptance server device 3 and the base server device 4.

[0073] Moreover, when the transmission processing unit 20b transmits the first information, the reception processing unit 20a receives, as display information, first display information indicating a response result for the above-mentioned predetermined plurality of operation events indicated by the first information from the image forming apparatus 1. The first display information is received as display information when the above-mentioned conditions are satisfied. Therefore, the reception of the first display information implemented by the reception processing unit 20a is also executed via the communication unit 24. In the above network structure example, the reception of such first display information by the reception processing unit 20a is also executed via the base server device 4 and the acceptance server device 3.

[0074] Processing example of the present system 100

[0075] As described above, the structures of the image forming apparatus 1 and the terminal device 2 have been described. In the above structure, it is possible to always transmit the first information including the above-mentioned predetermined plurality of operation events.

[0076] On the other hand, the terminal device 2 may also determine whether to transmit the first information based on the communication time condition, and transmit the first information only when the condition is satisfied. This communication time can refer to the time required to obtain display information such as an image for one operation event. In this case, the image forming apparatus 1 returns the first display information when the first information is received. On the other hand, when the condition is not satisfied, the terminal device 2 only needs to transmit each received operation event. In this case, for each received operation event, the image forming apparatus 1 does not return the first display information, but returns display information indicating a response result for each operation event.

[0077] In this example, the above-mentioned predetermined plurality of operation events transmitted by the terminal device 2 in a manner included in the first information are a plurality of operation events determined based on the following communication time for the communication with the image forming apparatus 1 implemented by the communication unit 24.

[0078] The communication time can refer to the time required from the transmission of an operation event to the reception of display information as a response to the operation event. For example, the communication time can also be set as the time required from the start of the operation event that requests the initial display information to the reception of the display information. The operation event that requests the initial display information can also be set as, for example, the operation event in which the user opens a web page for accessing the image forming apparatus 1 using a browser. In addition, the web page in the above network structure example only needs to be a web page for accessing the reception server apparatus 3 or a web page that designates the image forming apparatus 1 on the web page. Furthermore, the communication time can also be obtained, for example, at the time point when the terminal apparatus 2 starts or restarts communication with the image forming apparatus 1. Moreover, the transmission processing unit 20b can be configured to transmit the first information based on the communication time.

[0079] More specifically, the above-mentioned predetermined multiple operation events can be set as the multiple operation events received when, for the communication with the image forming apparatus 1 implemented by the communication unit 24, the communication time required from the transmission of the operation event to the reception of the display information as a response to the operation event is longer than the predetermined time. That is, when the time required to obtain the display information from the image forming apparatus 1 exceeds the predetermined time, the transmission processing unit 20b can aggregate the multiple operation events and transmit them to the image forming apparatus 1 side as the first information.

[0080] When observed from the image forming apparatus 1 side, the above-mentioned predetermined multiple operation events satisfy the following conditions and are received in a manner included in the first information. That is, the above-mentioned predetermined multiple operation events received by the image forming apparatus 1 in a manner included in the first information become the multiple operation events determined based on the communication time required from the transmission to the reception of the operation event performed using the terminal apparatus 2 for the communication with the terminal apparatus 2 implemented by the communication unit 14. Moreover, the reception processing unit 10b receives the first information based on the communication time. More specifically, the above-mentioned predetermined multiple operation events can also be set as the multiple operation events received when, for the communication with the terminal apparatus 2 implemented by the communication unit 14, the communication time is longer than the predetermined time.

[0081] Thus, in the terminal device 2, even when it is determined that it will take time to obtain the display information from the image forming device 1, the contents of a plurality of operation events generated by the user can be transmitted in a manner that aggregates them into one communication. Further, in such a case, in the image forming device 1, the response results for the above-described plurality of operation events can be replied in a manner that aggregates them into one communication as the response to the operation event. Thereby, the system 100 can reduce the number of communications to implement efficient communication.

[0082] The processing flow of the terminal device 2 accompanied by such a determination will be described using Figure 2 as follows. Figure 2 FIG. is a flowchart for explaining a processing example of the terminal device 2 in the system 100.

[0083] Figure 2 The processing example of

[0084] shows an example of an update process in which the operation image displayed by the image forming device 1 is periodically updated in the terminal device 2. Here, an example is cited in which the display information transmitted from the image forming device 1 is an operation image to be displayed on a browser that can be displayed on the display unit 22 of the terminal device 2.

[0085] In this update process, the terminal device 2 performs a loop process (steps S11s, S11e) on the processes shown in steps S12 to S16 at a certain cycle. This cycle is not limited, but can be set, for example, at intervals of 0.3 seconds to 0.5 seconds.

[0086] In this loop process, first, the reception processing unit 20a records the communication start time in the internal memory or the storage unit 21 (step S12). In step S12, for example, the communication start time can be recorded in advance as a variable on the browser. The communication start time can be recorded as the start time of an operation event generated by the user. Then, the reception processing unit 20a starts the communication using the communication unit 24 by instructing the communication unit 24 to perform communication with the image forming device 1 for obtaining the operation image (step S13).

[0087] When the image forming apparatus 1 has a response, that is, when it becomes YES in step S14, the reception processing unit 20a performs the following processing. That is, the reception processing unit 20a records the communication end time in the internal memory or the storage unit 21 (step S15). In step S15, for example, the communication end time can be pre-recorded as a variable on the browser. And, in this case, the reception processing unit 20a causes the operation image obtained by this response to be displayed on the browser (step S16). Of course, the order of step S15 and step S16 is not limited.

[0088] In the above processing example, the communication start time and the communication end time recorded in steps S12 and S15 can be used to obtain the time required for obtaining the operation image, that is, the communication time. As described above, this communication time can be used to determine whether to send the first information.

[0089] Next, using Figures 3 to 5 , as an example of a click operation implemented by a mouse, a more specific processing example will be described. The click operation implemented by the mouse includes an operation of depressing the click button of the mouse and an operation of raising the click button to restore it. For the purpose of description, the above operation of depressing the click button of the mouse is referred to as a mouse press operation, and the above operation of raising the click button of the mouse to restore it is referred to as a mouse release operation. The mouse press operation corresponds to an operation in which a touch is detected by a touch panel or a touch sensor in a touch operation. The mouse release operation corresponds to an operation in which, after the detection of the touch, the touch using the touch panel or the touch sensor is no longer detected in a touch operation. Therefore, the following description can also be applied to a touch operation.

[0090] First, using Figure 3 , a processing example when a mouse press operation occurs in the terminal device 2 in the present system 100 will be described. Figure 3 FIG. is a flowchart for explaining the processing example.

[0091] In the present system 100, the description will be made from the stage where the operation image is provided from the image forming apparatus 1 to the terminal device 2, that is, the stage where the remote control panel is started to be provided. First, the transmission processing unit 20b of the terminal device 2 registers a change point of an operation event that is an event indicating a user operation from the operation unit 23 (step S21). The change point registered here refers to the coordinates currently represented by the mouse and refers to the start point of the change. In addition, the coordinates can be set as, for example, the X coordinate and the Y coordinate in the operation image. Regarding this point, the same applies to the following coordinates.

[0092] Next, the transmission processing unit 20b refers to the registered change points to determine whether there is a change in the operation event generated by the user operation (step S22). If the result in step S22 is NO, it waits for a change in the operation event. If the result in step S22 is YES, the transmission processing unit 20b determines whether this change is caused by a mouse press operation, that is, whether a mouse press operation has occurred (step S23).

[0093] If the result in step S23 is YES, the transmission processing unit 20b records the coordinates pressed by the mouse (step S24), and returns to step S22. After step S24, no transmission to the image forming apparatus 1 side related to the operation event of this mouse press operation is performed, but it waits for a mouse release operation described later. In addition, if the result in step S23 is NO, the transmission processing unit 20b or the control unit 20 executes processing corresponding to other operation events detected in step S22, and returns to step S22.

[0094] Next, use Figure 4 And a processing example when a mouse release operation occurs in the terminal device 2 in the present system 100 will be described. Figure 4 It is a flowchart for explaining the processing example.

[0095] Here, also in the present system 100, the explanation starts from the stage where the operation image is provided from the image forming apparatus 1 to the terminal device 2, that is, the stage where the remote control panel starts to be provided. However, actually, it becomes the processing example through Figure 3 and the processing after the mouse press operation has been completed.

[0096] First, the transmission processing unit 20b of the terminal device 2 registers the change points of the operation event in the same manner as in step S21 (step S31). The change points registered here also refer to the coordinates currently indicated by the mouse and the change start point.

[0097] Next, the transmission processing unit 20b refers to the registered change points to determine whether there is a change in the operation event generated by the user operation (step S32). If the result in step S32 is NO, it waits for a change in the operation event. If the result in step S32 is YES, the transmission processing unit 20b determines whether this change is caused by a mouse release operation, that is, whether a mouse release operation has occurred (step S33).

[0098] When the condition in step S33 is yes, the transmission processing unit 20b records the coordinates where the mouse is lifted in a manner associated with the coordinates where the mouse is pressed and recorded in step S24 (step S34). Next, the transmission processing unit 20b generates communication content based on these coordinates (step S35). The generated communication content is set as the first information indicating the mouse click operation of the coordinates where the mouse is pressed or the coordinates where the mouse is lifted. Next, the transmission processing unit 20b starts communication for the mouse click as the communication content with respect to the image forming apparatus 1 side via the communication unit 24 (step S36).

[0099] Next, the reception processing unit 20a determines whether there is a reply of the first display information as a response from the image forming apparatus 1 side (step S37). When the condition in step S37 is no, it waits for the reply. When the condition in step S37 is yes, the reception processing unit 20a causes the operation image to be displayed on the browser based on the received first display information (step S38), and returns to step S32. In addition, when the condition in step S33 is no, the transmission processing unit 20b or the control unit 20 executes processing corresponding to other operation events detected in step S22, and returns to step S32.

[0100] In the example described using Figures 3 to 5 as the above-mentioned predetermined plurality of operation events, the first information includes a first event as an operation event for starting the specified selection and a second event as an operation event for ending the specified selection. The selection here can refer to the selection of the position of the operation image, and more specifically, can refer to the selection of the target located at the position included in the operation image.

[0101] As described in Figure 4 when the terminal device 2 transmits information indicating a mouse click operation as the first information, as described in step S37, the image forming apparatus 1 transmits the first display information.

[0102] Next, including such processing, an example of the processing when receiving a mouse click operation of the image forming apparatus 1 in the present system 100 will be described using Figure 5 as follows. Figure 5 It is a flowchart for explaining the processing example.

[0103] First, the reception processing unit 10b of the image forming apparatus 1 determines whether there is communication from the terminal device 2 (step S41). In the case where there is no communication, it waits for communication. In the case where there is communication, that is, when the result in step S41 is yes, the reception processing unit 10b analyzes the communication content (step S42). The reception processing unit 10b determines whether a mouse click operation has occurred in the terminal device 2 based on the analysis result (step S43).

[0104] When a mouse click operation has occurred, that is, when the result in step S43 is yes, the reception processing unit 10b notifies a mouse press and a mouse release for the UI image that is the operation image of the display unit 12 (steps S44, S45). In addition, the order of steps S44 and S45 is not limited.

[0105] The control unit 10 accepts this notification and updates the operation image based on the mouse click, and causes the display unit 12 to perform display (step S46). Next, the transmission processing unit 10a updates the operation image for the terminal device, which is the operation image that accepts this notification and is used for display by the terminal device 2, based on the mouse click (step S47). In addition, the order of steps S46 and S47 is not limited.

[0106] Next, the transmission processing unit 10a generates communication content in a manner that includes the updated operation image for the terminal device or first display information generated based on the operation image for the terminal device (step S48). Then, the transmission processing unit 10a transmits the generated communication content to the terminal device 2 side via the communication unit 14 (step S49), and returns to step S41. In addition, in the case where the result in step S43 is no, the transmission processing unit 10a or the control unit 10 performs processing corresponding to the result parsed in step S42, and returns to step S41.

[0107] In addition, as described in Figure 2 , the terminal device 2 can also perform image update periodically in the background so as to display in real time the change of the operation image of the image forming apparatus 1 even when there is no operation performed by the user. In the present system 100, it is also possible to aggregate the processing for this image update. Specifically, in the case where the timing of the image update occurs between the mouse press and the mouse release, in the present embodiment, for example, wait for the image update before the mouse release is completed, and by replacing it with the image update realized by the mouse click, it is possible to aggregate the processing of the image update also in the communication response.

[0108] Next, use Figure 6 and Figure 7, the overall processing example for mouse click operations in the present system 100 will be described while listing examples of operation images. Figure 6 It is a flowchart for explaining the processing example for mouse click operations in the present system 100. Figure 7 It is for showing Figure 6 A schematic diagram of the transition example of the display image in the processing example of

[0109] First, by receiving the provision of the remote control panel, the terminal device 2, for example, displays an operation image such as the operation image 60s of Figure 7 (step S61). The operation image 60s can include an icon group 61 showing various functions such as copying and scanning in the image forming device 1. In addition, the operation image 60s can also include a background image 62. In addition, when the operation image 60s is not the home page image, it can include a home page button 63 for returning to the home page image. In addition, the operation image 60s can also have a button 64 for transitioning to an operation image representing the printing task and the current state of the image forming device 1, a button 65 for performing interruption processing, a button 66 for selecting a media tray, and an information bar 67 for displaying various information. Of course, the example of the operation image 60s is not limited to this, and other buttons can also be displayed. Even if it is an image that does not include some or all of the buttons or icons shown, as long as it is an operation image. In addition, in Figure 7 , in the operation image 60s, the mouse pointer 68 is displayed at a predetermined position, for example.

[0110] Next, the terminal device 2 receives an operation on the operation image 60s by the operation unit 23, and thus requests the reception server device 3 to obtain the next operation image, that is, the operation image reflecting this operation (step S62). In Figure 7 , in the operation image 60s, an operation is received from the state where the mouse pointer 68 is displayed. Hereinafter, in step S62, an example where the icon 61a of the copy function is clicked as an operation will be described. In this case, in step S62, the operations of pressing and lifting the mouse are aggregated into first information and included in the request for obtaining the operation image.

[0111] The reception server device 3 that has received the request for obtaining the operation image transmits this request to the base server device 4 (step S63). The base server device 4 that has received this request transmits this request to the image forming device 1 (step S64).

[0112] The image forming device 1 uses Figure 5The operation image for the terminal device 2 is updated in the manner exemplified, and is sent to the base server device 4 as the first display information (step S65). The base server device 4 that has received the first display information sends the first display information to the acceptance server device 3 (step S66). The acceptance server device 3 that has received the first display information sends the first display information to the terminal device 2 (step S67).

[0113] The terminal device 2 updates the operation image based on the received first display information (step S68). Through this update, the operation image 60s is transformed into an operation image 60e related to the copying function. In addition, while the pointer 68 is being displayed, the operation image 60e can include, for example, a basic setting button group 69a, an application setting button group 69b, and a button group 69c used when executing copying. In addition, the terminal device 2 records the communication time t1 required up to this update (step S69). In addition, the order of steps S68 and S69 is not limited.

[0114] Effects of this embodiment

[0115] According to this embodiment, when a part or all of the operation image of the image forming apparatus 1 is displayed via a network and operations are performed from the terminal device 2, the time required for reflecting user operation events can be shortened, thereby improving user convenience. This is because in this embodiment, user operation events and their response data can be aggregated into one communication to reduce the number of communications, enabling efficient communication.

[0116] Therefore, according to this embodiment, even in a network environment with a slow communication speed, such as when the communication path between the terminal device 2 and the image forming apparatus 1 is far regardless of their physical positions, the response speed of the remote control panel to user operations can be improved. In particular, according to this embodiment, without incurring hardware costs such as enhancing server devices and communication lines, the response speed of the remote control panel to user operations can be improved only by changing the programs of the terminal device 2 and the image forming apparatus 1.

[0117] Regarding the above effects, use Figure 8 and is described by comparison with a comparative example. Figure 8 It is a flowchart for explaining the processing during a mouse click operation in the image forming system according to the comparative example. The image forming system according to the comparative example is as Figure 8 shown, and instead of the terminal device 2 and the image forming apparatus 1, a terminal device 2c that does not have a function related to the transmission of the first information and an image forming apparatus 1c that does not have a function related to the transmission of the first display information are respectively provided.

[0118] In the comparative example, the terminal device 2c displays the operation image 60s in the same manner as in step S61 (step S71). Next, the terminal device 2c accepts a mouse press operation for the operation image 60s, sends this information to the acceptance server device 3, and requests a response therefrom (step S72). Thereafter, the image forming device 1c receives this request using the same path as Figure 6 the same. The image forming device 1c that has received this request implements a response to the received request via the base server device 4 and the acceptance server device 3 using the same path as Figure 6 the same. The communication time required from the start of step S72 until this response is defined as the communication time tcc.

[0119] Next, the terminal device 2c accepts a mouse release operation for the operation image, sends this information to the information acceptance server device 3, and requests a response therefrom (step S73). Thereafter, the image forming device 1c receives this request using the same path as Figure 6 the same. The image forming device 1c that has received this request implements a response to the received request via the base server device 4 and the acceptance server device 3 using the same path as Figure 6 the same. The communication time required from the start of step S73 until this response also becomes equal to the communication time tcc.

[0120] Next, the terminal device 2c requests the acceptance server device 3 to obtain the updated operation image (step S74). Thereafter, the image forming device 1c receives this request using the same path as Figure 6 the same. The image forming device 1c that has received this request implements a response to the received request via the base server device 4 and the acceptance server device 3 using the same path as Figure 6 the same. Since the communication time required from the start of step S74 until this response includes the operation image or information for its display, it becomes a communication time tci that is longer than the communication time tcc.

[0121] The terminal device 2c updates the operation image based on the received response (step S75). Through this update, the operation image 60s is transformed into an operation image 60e related to the copying function.

[0122] Thus, in the image forming system according to the comparative example, from the moment a user clicks the mouse on the remote control panel until the next operation image is displayed, a total of three communications of mouse press, mouse release, and image acquisition are performed internally. Therefore, in the comparative example, a total time of 2×tcc + tci is required before switching the operation image by updating. This total becomes 5 seconds or more when measured based on a predetermined system environment. Here, an example of the predetermined system environment is a very long communication path where the communication path between the terminal device 2c and the image forming device 1c is from Japan via an overseas reception server device 3 and a base server device 4.

[0123] In contrast, in the present embodiment, the operations of mouse press and mouse release are aggregated. As a result, in the present embodiment, when measured based on the same system environment as the above-mentioned predetermined system environment, the response time of the remote control panel to user operations is approximately 3.0 to 3.2 seconds, improving the response speed by approximately 40%. Thus, in the present embodiment, an efficient communication method can be provided in the remote control panel function as a remote operation function of the image forming device.

[0124] Next, the application scenarios of the present system 100 will be described, and the cases where the above effects are beneficial effects will be described. The provision of the remote control panel can be used, for example, by an operator located at a relatively far distance from the image forming device 1. Examples of such an operator include an IT (Information Technology) manager or service personnel, a dealer engineer, etc. These operators can perform troubleshooting or training for end users without having to go to the site where the image forming device 1 is installed. For example, as scenarios for using the remote control panel, the following scenarios (1) and (2) can be envisioned.

[0125] (1) An IT manager within the company where the image forming device 1 is installed uses the terminal device 2 and directly accesses the image forming device 1 from within the same internal network for use.

[0126] (2) A service personnel or dealer engineer outside the company where the image forming device 1 is installed uses the terminal device 2 and accesses the image forming device 1 via the reception server device 3 and the base server device 4 for use.

[0127] In the present embodiment, in either of the scenarios (1) and (2) like this, the operator can comfortably and quickly perform processing by using the present system 100 with a relatively fast response speed of the remote control panel.

[0128] Moreover, in this embodiment, by adopting the communication-time-based determination as described above, it is possible to perform the process of aggregating multiple operation events only when the communication time is slow. The specific description of this effect is as follows.

[0129] First, as described in Figure 2 , in order to display the changes in the operation image of the image forming apparatus 1 in real time even when there is no operation performed by the user, the terminal device 2 can also perform image update periodically in the background. The terminal device 2 can also record in advance the communication time required for this periodic image update process. Moreover, by adopting the above determination, when a user operation such as a mouse click occurs on the browser, in a case where it is determined based on the recorded communication time that the communication speed is slow, for example, the communication time is 0.5 seconds or more, the communication content is switched and aggregated into one communication. Specifically, the operation events of mouse press and mouse release are aggregated into a mouse click as one event and transmitted to the image forming apparatus 1 side. Therefore, it is possible to perform such aggregation of operation events only in an environment with a slow communication speed. For example, in an environment with a fast communication speed, such aggregation is not performed, and even if there is an operation event requesting an image change that cannot be represented by aggregation, such an image change can be reflected.

[0130] Application example: Example of processing during icon movement operation

[0131] The system 100 is not limited to mouse clicks and can also be applied to other multiple operation events. Here, an example of a movement operation of an object such as an icon, that is, a drag operation, will be described. The icon movement operation can also be said to be an example of a range specification operation. Therefore, the following description can also be applied to range specification operations other than the movement operation of an object such as an icon, for example, an operation of selecting multiple files from a file list included in the operation image. First, such a range specification operation will be described.

[0132] Range specification operations such as icon movement operations can be defined by operations including a third event that is an operation event for starting the specification of a selection range and a fourth event that is an operation event for expanding the selection range. In this case, the first information transmitted by the terminal device 2 to the image forming apparatus 1 side includes the third event and the fourth event as the above-mentioned predetermined multiple operation events. In the case of a movement operation of an icon or the like, the third event corresponds to a mouse press operation, the fourth event corresponds to a mouse movement operation, and the first information includes the operation events of the mouse press operation and the mouse movement operation.

[0133] In addition, the range specifying operation can be defined by an operation that includes a fourth event as an operation event for expanding the selection range and a fifth event as an operation event for specifying the end of the selection range. In this case, the first information includes the fourth event and the fifth event as the above-mentioned predetermined multiple operation events. In the case of a moving operation of an icon or the like, the fourth event corresponds to a mouse movement operation, the fifth event corresponds to a mouse lift operation, and the first information includes the operation events of the mouse movement operation and the mouse lift operation.

[0134] In addition, when summarizing the fourth event and the fifth event as described above, it is necessary to detect the fourth event. In this case, the terminal device 2 only needs to apply a thinning process for thinning out a part of the operation events from a series of operation events received by the operation unit 23. For example, when the amount of movement of the mouse reaches a certain amount, the thinning process can be set to thin out the operation events during the period up to that point. Thus, the fourth event to be summarized with the fifth event can be detected as a movement event completed immediately before receiving the fifth event.

[0135] When applying such a thinning process, the above-mentioned predetermined multiple operation events can also be defined as multiple operation events obtained by thinning out a part of a series of operation events received by the terminal device 2. In addition, such a thinning process is not limited to the application example described here, and can be similarly applied in the processes described with reference to Figures 1 to 7 the processes described.

[0136] Use Figures 9 to 11 to illustrate a processing example for the icon movement operation in the present system 100. Figure 9 FIG. is a flowchart for illustrating a processing example for the icon movement operation in the present system 100. Figure 10 For indicating Figure 9 a schematic diagram of a transition example of the display image in the processing example of. Figure 11 For indicating Figure 9 the display image in the processing example of, subsequent to Figure 10 a schematic diagram of a transition example.

[0137] First, similar to step S61 of Figure 6 , the terminal device 2 displays, for example, an operation image such as the operation image 60a of Figure 10 (step S91). In addition, the operation image 60a is illustrated with the same image as the operation image 60s. In Figure 10 , in the operation image 60a, the mouse pointer 68, for example, accepts an operation from a state where it is displayed at a predetermined position.

[0138] Next, the terminal device 2 receives a mouse press operation and a mouse movement operation on the operation image 60a through the operation unit 23, and thus requests the reception server device 3 to acquire the next operation image, that is, the operation image reflecting this operation (step S92). In this case, in step S92, the operations of mouse press and mouse movement are aggregated as first information and included in the acquisition request of the operation image. Here, an example in which a mouse press operation is completed on the icon 61a for the copy function will be described.

[0139] Regarding the request for acquisition of this operation image, similar to steps S63 and S64, the image forming device 1 receives this request (steps S93, S94). The image forming device 1 updates the operation image for the terminal device 2 in a manner corresponding to the mouse press and mouse movement operations, and similar to steps S65 to S67, sends the first display information to the terminal device 2 (steps S95 to S97).

[0140] The terminal device 2 updates the operation image based on the received first display information (step S98). Through this update, the operation image 60a is transformed into an operation image 60b in which the icon 61a for the copy function and the pointer 68a are respectively moved to the icon 61b and the pointer 68b.

[0141] Next, the terminal device 2 requests the reception server device 3 to acquire the next operation image by continuing the movement operation of moving the icon 61b on the operation image 60b (step S99). In this case, it is also possible to aggregate the remaining multiple movement operations in the thinning process as first information and include it in the acquisition request of the operation image.

[0142] Regarding the request for acquisition of this operation image, similar to steps S93 and S94, the image forming device 1 receives this request (steps S100, S101). The image forming device 1 updates the operation image for the terminal device 2 in a manner corresponding to the mouse movement operation, and similar to steps S95 to S97, sends the first display information to the terminal device 2 (steps S102 to S104).

[0143] The terminal device 2 updates the operation image based on the received first display information (step S105). Through this update, the operation image 60b is transformed into an operation image 60c in which the icon 61b for the copy function and the pointer 68b are respectively moved to the icon 61c and the pointer 68c. Figure 11 of the operation image 60c.

[0144] Next, the terminal device 2 performs the same processing as in steps S99 to S105 (steps S106 to S112) by continuing the moving operation of moving the icon 61c for the operation image 60c. As a result, the operation image 60c is transformed into an operation image (not shown).

[0145] Next, after the terminal device 2 continues the moving operation of moving the icon corresponding to the icon 61c for the operation image and then performs a mouse lift operation, it requests the reception server device 3 to obtain the next operation image (step S113). In this case, in step S113, the operations of mouse movement and mouse lift are aggregated as the first information and included in the request for obtaining the operation image.

[0146] For the request for obtaining the operation image, in the same manner as in steps S93 and S94, the image forming device 1 receives the request (steps S114 and S115). The image forming device 1 updates the operation image for the terminal device 2 in a manner corresponding to the mouse movement and mouse lift operations, and in the same manner as in steps S95 to S97, sends the first display information to the terminal device 2 (steps S116 to S118).

[0147] The terminal device 2 updates the operation image based on the received first display information (step S119). Through this update, the operation image 60c is transformed into an operation image 60d in which the icon 61c of the copying function and the pointer 68c are respectively moved to the icon 61d and the pointer 68d. In particular, since in the operation image 60c, the icon 61c reaches the right end of the operation image 60c, in the transformed operation image 60d, other icons of the icon group 61 also move to the left. At this time, the background image 62 can also be moved to the left together with the above other icons. In addition, the operation image displayed in step S112 becomes an image between the operation image 60c and the operation image 60d. In addition, as Figure 11 shown, the communication time required for each communication is represented as t1, and the communication time required for a series of communications becomes 4 × t1. Figure 9 shown, the communication time required for each communication is represented as t1, and the communication time required for a series of communications becomes 4 × t1.

[0148] In addition, Figure 10 and Figure 11 The transformation examples shown are just examples. For example, it is also possible to move other icon groups to the left direction in advance before the selected icon has been completely moved to the right end of the screen by mouse movement. That is, although in Figure 10 and Figure 11In the example, an example is given where the above-mentioned other icons are also moved at the stage where the icon of the moving object reaches one end of the operation image. However, it is also possible to move the above-mentioned other icons in the opposite direction as the icon of the moving object moves before reaching one end.

[0149] Regarding the effects of the application example, use Figure 12 and Figure 13 and it will be described by comparison with the comparative example. Figure 12 FIG. is a flowchart for explaining the processing when an icon moving operation is performed in the image forming system according to the comparative example. Figure 13 is for showing Figure 12 a schematic diagram of a transition example of the display image in the processing of

[0150] In the comparative example, the terminal device 2c, in the same manner as in step S91, displays the operation image 60a shown in Figure 10 and Figure 13 (step S121). Next, the terminal device 2c accepts a mouse press operation on the operation image 60a, sends this information to the acceptance server device 3, and requests a response (step S122). Thereafter, the image forming device 1c receives this request through the same path as Figure 9 . The image forming device 1c that has received this request implements a response to the received request through the same path as Figure 9 via the base server device 4 and the acceptance server device 3. Let the communication time required from the start of step S122 to this response be the communication time tcc.

[0151] Next, the terminal device 2c performs a mouse lift movement on the operation image 60a, sends this information to the acceptance server device 3, and requests a response (step S123). Thereafter, the image forming device 1c receives this request through the same path as Figure 9 . The image forming device 1c that has received this request implements a response to the received request through the same path as Figure 9 via the base server device 4 and the acceptance server device 3. Next, the same mouse movement operation and its response are performed in the same manner (steps S124, S125).

[0152] Next, the terminal device 2c accepts a mouse lift operation on the operation image 60a, sends this information to the acceptance server device 3, and requests a response (step S126). Thereafter, the image forming device 1c receives this request through the same path as Figure 9 . The image forming device 1c that has received this request Figure 9For the same path, the response to the received request is implemented via the base server device 4 and the reception server device 3.

[0153] Next, the terminal device 2c requests the reception server device 3 to obtain the updated operation image (step S127). Thereafter, using the same path, the image forming device 1c receives this request. The image forming device 1c that has received this request uses the same path and implements the response to the received request via the base server device 4 and the reception server device 3. Figure 9 For the same path, the image forming device 1c receives this request. The image forming device 1c that has received this request uses the same path and implements the response to the received request via the base server device 4 and the reception server device 3. Figure 9 For the same path, the response to the received request is implemented via the base server device 4 and the reception server device 3.

[0154] The terminal device 2c updates the operation image based on the received response (step S128). Through this update, the operation image 60a is transformed into the operation image 60d. However, as illustrated by, when continuous movement operations occur, the image update process is postponed. Figure 13 of the operation image 60d. However, as illustrated by Figure 12 when continuous movement operations occur, the image update process is postponed.

[0155] As Figure 13 shown, in the image forming system according to the comparative example, when the user presses and moves the selection icon 61a on the remote control panel for the operation image 60a with a mouse, the operation image 60d is immediately displayed. That is, in the comparative example, the operation images 60b and 60c illustrated by and are not experienced, and the operation image 60a is immediately switched to the operation image 60d. This is because in the comparative example, a total of six communications of mouse press, mouse move, mouse move, mouse move, mouse lift, and image acquisition are implemented internally. Therefore, before switching the operation image by update, a total time of 5×tcc + tci is required. This total becomes 11 seconds or more when actually measured based on the above-mentioned predetermined system environment. Figure 10 and Figure 11 This is because in the comparative example, a total of six communications of mouse press, mouse move, mouse move, mouse move, mouse lift, and image acquisition are implemented internally. Therefore, before switching the operation image by update, a total time of 5×tcc + tci is required. This total becomes 11 seconds or more when actually measured based on the above-mentioned predetermined system environment.

[0156] In contrast, in the application example of the present embodiment, operations are aggregated in a manner such as mouse press and mouse move, mouse move and mouse lift. Thus, in the present embodiment, when actually measured based on the same system environment as the above-mentioned predetermined system environment, the response time of the remote control panel to the user operation is about 8.0 to 8.4 seconds, improving the response speed by about 30%. In this way, in the application example of the present embodiment, an efficient communication method can be provided in the remote control panel function as the remote operation function of the image forming device. And, in this application example, since the operation image is changed step by step compared with the comparative example, the convenience for the user is also increased.

[0157] As described in the above application example, the present system 100 can also be applied to the case where a user performs a drag operation on a remote control panel. Moreover, in the present embodiment, the responsiveness of the drag operation can also be improved by such an application.

[0158] Another network structure example of the image forming system

[0159] Use Figure 14 Another structural example of the image forming system according to the present embodiment will be described. Figure 14 It is a schematic block diagram showing another structural example of an image forming system including the image forming apparatus and the terminal apparatus according to the present embodiment.

[0160] The image forming system according to the present embodiment can be configured as an image forming system 100A having a network structure example as shown in Figure 14 . The image forming system 100A connects the terminal apparatus 2 and the image forming apparatus 1 using a LAN (Local Area Network), and connects a communication control apparatus 5 such as a router to the LAN. This LAN can be, for example, the LAN of a company where the image forming apparatus 1 is installed. The communication control apparatus 5 is a device that controls communication between devices in the LAN and the network N. In the network N1, server devices 6 such as an acceptance server device 3 and a base server device 4 are connected, for example. Of course, the network structure applicable in the present embodiment is not limited to the network structure exemplified by Figure 1 and Figure 14 .

[0161] (Other modification examples)

[0162] The present invention is not limited to the above-described embodiment, and can be appropriately modified without departing from the gist. For example, the image forming apparatus applied to the image forming system according to the present embodiment can be widely applied to a copying machine, a facsimile machine, a multifunction machine having these functions, etc., as can be understood from the operation image etc. of Figure 7 .

[0163] In addition, each device provided in the image forming system according to the above-described embodiment can have, for example, the following hardware structure. Figure 15 It is a diagram showing an example of the hardware structure of the device.

[0164] Figure 15 The device 1000 shown can include a processor 1001, a memory 1002, and an interface 1003. The interface 1003 can include, for example, a communication interface required according to the device, or an interface with a sensor, an input / output device, etc.

[0165] The processor 1001 can also be, for example, a CPU, a GPU (Graphics Processing Unit), an MPU (Micro Processor Unit) also known as a microprocessor, etc. The processor 1001 can also include multiple processors. The memory 1002 is constituted, for example, by a combination of a volatile memory and a non-volatile memory. The functions in each device are realized by the processor 1001 reading the program stored in the memory 1002 and executing the program while exchanging the required information via the interface 1003.

[0166] In addition, the above program includes a set of commands (or software code) for causing a computer to implement one or more of the functions described in the embodiments when read by the computer. The program can also be stored in a non-temporary computer-readable medium or a tangible storage medium. As non-limiting examples, the computer-readable medium or the tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), or other memory technologies. In addition, as non-limiting examples, the computer-readable medium or the tangible storage medium includes CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) Blu-ray disc, or other optical disc storage devices, cassette tapes, magnetic tapes, magnetic disk storage devices, or other magnetic storage devices. The program can also be transmitted on a temporary computer-readable medium or a communication medium. As non-limiting examples, the temporary computer-readable medium or the communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0167] Although the present invention has been described above according to the above embodiments, the present invention is not limited only to the structures of the above embodiments, and of course includes various deformations, corrections, and combinations that can be made by those skilled in the art within the scope of the invention of the claims in the claims of this application.

[0168] Reference Signs

[0169] 1…Image forming apparatus; 2…Terminal device; 3…Receiving server device; 4…Basic server device; 5…Communication control device; 6…Server device; 10…Control unit; 10a…Transmission processing unit; 10b…Receiving processing unit; 11…Storage unit; 12…Display unit; 13…Operation unit; 14…Communication unit; 15…Printing unit; 20…Control unit; 20a…Receiving processing unit; 20b…Transmission processing unit; 21…Storage unit; 22…Display unit; 23…Operation unit; 24…Communication unit; 61a, 61b, 61c, 61d, 61e, 61s…Operation images; 61…Icon group; 61a, 61b, 61c, 61d…Icons for copying function; 62…Background image; 63…Home button; 64, 65, 66…Buttons; 67…Information bar; 68, 68a, 68b, 68c, 68d…Pointers; 1000…Device; 1001…Processor; 1002…Memory; 1003…Interface; N…Network; N1…Network.

Claims

1. An image forming device comprising: Ministry of Communications; A display unit that displays an operation image; a transmission processing unit that transmits display information obtained based on the operation image displayed on the display unit to a terminal device via the communication unit; a reception processing unit that receives the operation event received by the terminal device from the terminal device via the communication unit, When the reception processing unit receives first information including a plurality of predetermined operation events, the transmission processing unit transmits first display information indicating a response result to the plurality of predetermined operation events indicated by the first information to the terminal device as the display information.

2. The image forming apparatus according to claim 1, wherein: The first information includes, as the plurality of predetermined operation events, a first event which is an operation event for starting a selected designation and a second event which is an operation event for ending the selected designation.

3. The image forming apparatus according to claim 1, wherein: The first information includes, as the plurality of predetermined operation events, a third event that is an operation event for starting designation of a selection range, and a fourth event that is an operation event for expanding the selection range.

4. The image forming apparatus according to claim 1, wherein: The first information includes, as the plurality of predetermined operation events, a fourth event that is an operation event for expanding a selection range, and a fifth event that is an operation event for ending designation of the selection range.

5. The image forming apparatus according to any one of claims 1 to 4, wherein: The predetermined plurality of operation events are a plurality of operation events obtained by thinning out a part of a series of operation events received by the terminal device.

6. The image forming apparatus according to any one of claims 1 to 4, wherein: The transmission processing unit transmits the display information as information to be displayed on a browser of the terminal device.

7. The image forming apparatus according to any one of claims 1 to 4, wherein: The communication unit communicates with the terminal device via at least one information processing device.

8. A terminal device comprising: Operations Department; Ministry of Communications; a reception processing unit that receives display information obtained based on an operation image displayed on a display unit of the image forming device from the image forming device via the communication unit; a display unit that displays an operation image based on the display information received by the reception processing unit; a transmission processing unit that transmits the operation event received by the operation unit to the image forming device via the communication unit, The transmission processing unit transmits first information including a plurality of predetermined operation events received by the operation unit to the image forming apparatus. When the first information is transmitted by the transmission processing unit, the reception processing unit receives, as the display information, first display information indicating response results to the plurality of predetermined operation events indicated by the first information from the image forming apparatus.

9. The terminal device according to claim 8, wherein: The first information includes, as the plurality of predetermined operation events, a first event which is an operation event for starting a selected designation and a second event which is an operation event for ending the selected designation.

10. The terminal device according to claim 8, wherein: The first information includes, as the plurality of predetermined operation events, a third event that is an operation event for starting designation of a selection range, and a fourth event that is an operation event for expanding the selection range.

11. The terminal device according to claim 8, wherein: The first information includes, as the plurality of predetermined operation events, a fourth event that is an operation event for expanding a selection range, and a fifth event that is an operation event for ending designation of the selection range.

12. The terminal device according to any one of claims 8 to 11, wherein: The predetermined plurality of operation events are a plurality of operation events obtained by thinning out a part of a series of operation events received by the operation unit.

13. The terminal device according to any one of claims 8 to 11, wherein: The predetermined plurality of operation events are a plurality of operation events determined based on a communication time required from transmission of the operation event to reception of the display information as a response to the operation event, with respect to the communication between the communication unit and the image forming apparatus.

14. The terminal device according to any one of claims 8 to 11, wherein: The predetermined multiple operation events are multiple operation events that are accepted when the communication time required from the sending of the operation event to the reception of the display information as a response to the operation event is longer than the predetermined time for the communication between the communication unit and the image forming device.

15. The terminal device according to any one of claims 8 to 11, wherein: The reception processing unit receives the display information as information to be displayed by a browser of the terminal device.

16. The terminal device according to any one of claims 8 to 11, wherein: The communication unit communicates with the image forming apparatus via at least one information processing apparatus.

17. An operating method, wherein: executing a sending process in which an image forming device including a communication unit and a display unit for displaying an operation image sends display information obtained based on the operation image displayed on the display unit to a terminal device via the communication unit, executing a reception process in which the image forming device receives an operation event received by the terminal device from the terminal device via the communication unit, In the sending process, when the first information including a predetermined plurality of operation events is received by the receiving process, the first display information indicating the response result to the predetermined plurality of operation events indicated in the first information is sent to the terminal device via the communication unit as the display information.

Images