Information processing apparatus, information processing control method, and computer program

Abstract

To address an issue that, if a user forgets to disconnect a VNC connection with an apparatus and switches an application to a background state, the other users are unable to establish a VNC connection with the apparatus.SOLUTION: It is configured to allow other devices to establish a VNC connection even if an application transitions to a background, by canceling a priority state of the VNC connection.SELECTED DRAWING: Figure 6B

Description

Technical Field

[0001] It relates to a method for remotely operating an operation panel.

Background Art

[0002] In order to remotely operate the operation screen of an image processing apparatus, a method is disclosed in which by using VNC (Virtual Network Computing) using the RFB (Remote FrameBuffer) protocol, the operation screen of the image processing apparatus can be operated from a PC, a mobile terminal, etc. Specifically, an image processing apparatus having a VNC server function transmits screen information of the operation screen to a terminal having a VNC client function. The terminal displays the same screen as the operation screen displayed on the operation panel of the image processing apparatus based on the received screen information. By operating on the screen displayed on the terminal, the user can log in to the operation screen provided by the image processing apparatus or cause the image processing apparatus to execute processing without operating the operation panel of the image processing apparatus.

[0003] Patent Document 1 discloses a method in which when remotely operating an image processing apparatus using VNC, if the main body operation unit of the image processing apparatus does not receive an operation for a certain period of time, the image processing apparatus shifts to a power saving mode.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Devices that provide a remote operation function often limit the number of devices that can be remotely operated to one in order to limit the number of operators to one. When remotely operating an image processing device from a terminal using VNC or the like, if a certain user makes a VNC connection to the image processing device, other users cannot make a VNC connection to the image processing device. For example, a certain user makes a VNC connection from a terminal to an image processing device and logs in to the image processing device and performs a printing process by operating the terminal. Then, if the user forgets to disconnect the VNC connection and migrates the application that operates as a VNC client on the terminal to the background because the process has ended, the VNC connection continues, so other users cannot make a VNC connection to the image processing device. Therefore, the present invention provides a method by which another user can remotely operate the device even when the remote operation function has not been stopped despite the user having finished using the remote operation function of the device.

Means for Solving the Problems

[0006] A program for causing a computer of an information processing device that communicates with an image processing device to execute a control method, wherein the control method is a screen corresponding to a screen to be displayed on a display unit of the image processing device, and a display step of displaying a screen that is display-controlled by the image processing device; When the program stops operating as an operation target of a user on the information processing device and satisfies a predetermined condition, a transmission step of transmitting an instruction to stop the display control to the image processing device. When a predetermined connection setting between the image processing apparatus and the information processing apparatus is the first setting, the predetermined condition is that a predetermined time has elapsed in a state where the program has stopped operating as an operation target of the user on the information processing apparatus; when the predetermined connection setting is the second setting, the predetermined condition is that the predetermined time has elapsed without the program operating as an operation target of the user on the information processing apparatus in a state where the program has stopped operating as an operation target of the user A program characterized by the above.

Effects of the Invention

[0007] Even when the user has finished using the remote operation function of the device and the remote operation function has not been stopped, another user can remotely operate the device.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0009] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of the features described in the embodiments are essential for the solution means of the invention.

[0010] (Example 1) <System Configuration in this Example> First, the system configuration according to this example will be described with reference to FIG. 1. Here, the mobile terminal 100 and the MFP 110 will be described as examples of information processing apparatuses, but the apparatuses included in the information processing system are not limited to these. The mobile terminal 100 and the MFP 110 are connected to a LAN (Local Area Network) and can exchange information with each other. The mobile terminal 100 is connected to the LAN via an access point 120.

[0011] The mobile terminal 100 has a VNC (Virtual Network Computing) client function for remotely operating the MFP 110. Specifically, the mobile terminal 100 has VNC client software compatible with the RFB (Remote Frame Buffer) protocol. The mobile terminal 100 establishes a VNC connection to the MFP 110 and displays the screen shown on the operation panel of the MFP 110 on the mobile terminal 100. The MFP 110 has a VNC server function and, when a VNC connection is made from the mobile terminal 100, transmits screen information to the mobile terminal 100 which is the VNC client. The mobile terminal 100 draws the screen based on the transmitted screen information. Screen linkage is performed so that the screens displayed on the display units of the mobile terminal 100 and the MFP 110 are changed according to the operations received on the drawn screen and the operations received on the MFP 110.

[0012] Next, the hardware configuration of the mobile terminal 100 will be described with reference to FIG. 2. Note that although the mobile terminal 100 of the present embodiment assumes a device such as a smartphone or a tablet PC, other devices may be used as long as they are information processing devices capable of communicating with the MFP 110.

[0013] The CPU 201 reads out the control program stored in the ROM 202 and executes various processes for controlling the operation of the mobile terminal 100. The ROM 202 stores the control program. The RAM 203 is used as a temporary storage area such as the main memory and work area of the CPU 201. The storage 204 stores various data such as photos and electronic documents, for example, using eMMC, SSD, HDD, etc. Also, the OS (Operating System) 350 and the MFP application 300, which will be described later, are also stored in the storage 204. The RTC (Real Time Clock) 205 measures time.

[0014] In the case of the mobile terminal 100, it is assumed that one CPU 201 executes each process shown in the flowchart described later, but other modes may be used. For example, a plurality of CPUs may cooperate to execute each process shown in the flowchart described later.

[0015] The operation panel 206 has a touch panel function capable of detecting a user's touch operation, and displays various screens provided by the OS 350 and the MFP application 300. By inputting a touch operation to the operation panel 206, the user can input a desired operation instruction to the mobile terminal 100.

[0016] The speaker 207 and the microphone 208 are used when the user makes a call, for example, with another mobile terminal or a landline phone. The camera 209 performs imaging in response to a user's imaging instruction. The photo taken by the camera 209 is stored in a predetermined area of the storage 204. The wireless communication unit 211 executes wireless communication such as wireless communication LAN.

[0017] Next, the software configuration of the mobile terminal 100 will be described with reference to FIG. 3. FIG. 3 is a functional block diagram of software realized by the CPU 201 reading control programs stored in the ROM 202 and the storage 204.

[0018] The OS 350 is software for controlling the operation of the entire mobile terminal 100. Various applications can be installed in the mobile terminal 100, including the MFP application 300 described later. The OS 350 exchanges information with these applications, and changes the screen to be displayed on the operation panel 206 according to the instructions received from the applications. In addition, the OS 350 includes a group of device drivers for controlling various hardware, and provides APIs for using various hardware to applications operating on the OS. In this embodiment, for example, the wireless LAN control unit 351 corresponds to the group of device drivers. The wireless LAN control unit 351 is a device driver for controlling the wireless communication unit 211.

[0019] The MFP application 300 is an application installed on the mobile terminal 100. Printing instructions and scanning instructions can be sent from the MFP application to the MFP 110. Furthermore, as described above, it operates as a VNC client, has the function of displaying the operation screen of the MFP 110, and remotely operating the MFP 110. Various other applications may be installed on the mobile terminal 100 in addition to the MFP application 300, but the description thereof is omitted here.

[0020] The configuration of the MFP application 300 will be described in more detail. The screen control unit 301 controls the screen to be displayed on the operation panel 206 via the OS 350. The screen of the MFP application is displayed on the operation panel 206 by the screen control unit 301. Also, the screen control unit 301 receives operation instructions input by the user via the operation panel 206. The communication unit 302 controls the wireless communication by the wireless communication unit 211 via the OS 350, and performs transmission and reception of information with external devices such as the MFP 110.

[0021] The device search control unit 311 generates search data for searching for external devices connected to the mobile terminal 100, and transmits it to devices on the LAN 10 by the wireless communication unit 211. Then, it receives a response from the external device that responds to the corresponding search. The search data may be broadcast to all external devices on the LAN 10, or may be transmitted to a single external device by specifying an address.

[0022] The print job generation unit 305 generates a print job. The print job generated by the print job generation unit 305 is transmitted to the MFP 110 by the wireless communication unit 211, and the MFP 110 executes printing in response to the received print job.

[0023] The scan job control unit 307 transmits a scan instruction to the MFP 110 via the wireless communication unit 211. The MFP 110 executes a scan based on the instruction and transmits the data (scan data) generated by the scan to the mobile terminal 100. The mobile terminal 100 displays the received scan data. When saving the scan data, it is stored in the storage unit 306. The stored scan data is managed by the document management unit 308.

[0024] The remote operation unit 313 displays a screen based on the screen information acquired from the MFP 110 connected by VNC and accepts operations on the displayed screen. That is, it operates as a VNC client. Specifically, it displays a screen based on the screen information received from the VNC server and transmits operation information including the position and type of the operation received on the displayed screen to the MFP 110. The MFP 110 changes the screen to be displayed on the MFP 110 and the screen displayed on the mobile terminal 100 based on the received operation information. The application storage unit 306 temporarily stores various information generated by the MFP application 300. The timing unit 304 measures time.

[0025] FIG. 4 shows an example of the hardware configuration of the MFP 110. The MFP 110 includes a CPU 3020 that executes a program stored in the ROM 3030 or a large-scale storage device 3110 such as a hard disk. The CPU 3020 comprehensively controls each device connected to the system bus 3010.

[0026] 3040 is a RAM and functions as the main memory, work area, etc. of the CPU 3020. 3060 is an external input controller (PANELC) that controls instruction inputs from various hard keys 3120 or the touch panel (PANEL) 3070 provided in the multifunction machine. 3080 is a display controller (DISPC) that controls the display of a display module (DISPLAY) 3090 composed of, for example, a liquid crystal display. 3110 is a disk controller (DKC) that controls the large-scale storage device (HD) 3110.

[0027] Using 900 in FIG. 9, the operation unit of the MFP 110 will be described. A screen is displayed on the DISPLAY 3090, and the hard keys 3120 are arranged next to the DISPLAY 3090. User operations can be received by both the touch panel 3070 on the DISPLAY 3090 and the hard keys 3120. The logout key 901, which is one of the hard keys 3120, is a button for giving an instruction for logout processing to the MFP 110 when pressed while the user is logged in to the MFP 110.

[0028] 3050 is a network interface card (NIC) that exchanges data bidirectionally with an external device (e.g., the mobile terminal 100) or a file server via the network 10. 3400 is a wireless communication module that can connect to an access point or operate in access point mode to operate the MFP 110 as an access point and directly establish a wireless communication connection with the mobile terminal 100. 3200 is a printer, which is a printing unit on paper realized by an electrophotographic method. The printing method does not necessarily have to be an electrophotographic method. 3300 is a scanner, which is an image reading unit for reading an image printed on paper. In many cases, an ADF (auto document feeder) (not shown) is attached to the image reading unit 3300 as an option, and multiple sheets of originals can be automatically read. Note that the large-scale storage device 3110 may also be used as a temporary storage location for images in some cases.

[0029] FIG. 5 is a block diagram showing the software configuration of the MFP 110. This software is stored in, for example, the ROM 3030 and executed by the CPU 3020.

[0030] The UI processing unit 510 analyzes the input from the external input controller 3060 or the operation information received from the operation information receiving unit 522 described later. When a screen change is necessary, it generates screen data and displays it on the DISPLAY 3090.

[0031] On DISPLAY3090, various screens are displayed as shown in FIG. 9. For example, an ID card login screen 920, a keyboard login screen 910, a menu screen 930 displayed during login, an error screen (not shown) displayed when an error occurs, a counter screen (not shown) that displays counter information, etc. are displayed. When MFP110 is in the sleep state, DISPLAY3090 is in the non-display state.

[0032] The ID card login screen 920 is a screen for logging in to MFP110 with an ID card associated with user information. By holding the ID card over the ID card reader of MFP110 while this screen is being displayed, the user information is read and the login process is performed. When the keyboard login screen switching button 921 is pressed, the display switches to the keyboard login screen 910.

[0033] The keyboard login screen 910 is a screen for logging in to MFP110 by entering user information using the keyboard. Using the hard key 3120 or a soft keyboard (not shown), enter the username input field 911 and the password input field 912, and press the login button 913 to perform the login process. When the ID card login switching button 914 is pressed, the display switches to the ID card login screen 920.

[0034] The menu screen 930 is a menu screen that is displayed after logging in to MFP110 via the keyboard login screen 910 or the ID card login screen 920. Depending on the user's needs, the user can use various functions of MFP110 by pressing the copy button 931, the scan button 932, or the print button 933. When the user presses the logout button 934, the logout process is performed.

[0035] The remote operation processing unit 520 includes modules such as a connection processing unit 523, an operation information receiving unit 522, and a screen transmitting unit 521. The remote operation processing unit 520 is a VNC server module that performs communication according to the RFB protocol, and is a software module that communicates with the mobile terminal 100 operating as a VNC client via a network. The connection processing unit 523 performs connection processing when receiving a connection request from the MFP 110 or the mobile terminal 100 which is a VNC client, and performs disconnection processing when receiving a disconnection request for the connection.

[0036] The operation information receiving unit 522 receives operation information of the screen from the mobile terminal 100 and the MFP 110, and notifies the UI processing unit 510. The screen transmitting unit 521 is a display control unit that reads the screen data generated by the UI processing unit 510 from the RAM 3040 and performs display control of the screen by transmitting it to the mobile terminal 100 which is a VNC client.

[0037] The login processing unit 530 is a module that performs the login processing of the MFP 110. Based on user information and the like received from the UI processing unit, it performs user authentication, and when the user authentication is successful, it performs login. Also, when receiving an instruction to log out from the UI processing unit 510 or the like, it performs logout processing. The network control unit 500 is a module for controlling the network I / F 3050. Although not shown in the figure, there are also a printer module for controlling the printer 3200 and a scanner module for controlling the scanner 3300.

[0038] FIG. 8 is an example of a screen displayed on the operation panel 206 of the mobile terminal 100. When the mobile terminal 100 is activated, the home screen 810 is displayed. The MFP application icon 811 is an icon for starting the MFP application 300. When the MFP application icon 811 is operated, the MFP application 300 is started.

[0039] When the MFP application 300 is launched, the application home screen (device not selected) 820 is displayed. The selected device icon 821 displays information about the MFP with which the MFP application 300 is currently communicating and the MFP to which jobs are to be sent (hereinafter referred to as the target MFP). If the target MFP is not set, "No Printer" is displayed. The device search button 823 is a button for searching for the target MFP. The remote operation button 822 is a button for connecting to the target MFP via VNC and remotely operating the target MFP. In addition to these, the application home screen 820 may also display a print button 824 for sending a print job to the target MFP and a button for giving a scan instruction.

[0040] When the device search button 823 is pressed, a search for MFPs existing in the network is executed, and as a search result, the device search screen 830 is displayed. The device search screen 830 displays a list of MFPs that responded to the search. For example, by selecting a search result 831 that is one of the search results, the target MFP is selected.

[0041] When the target MFP is selected, the application home screen (device selected) 840 is displayed. The selected device icon 821 displays information about the MFP selected on the device search screen 830. When the user taps the remote operation button 822 with the target MFP selected, the remote operation function of the target MFP is started. Here, the following explanation will be given assuming that the target MFP is MFP110.

[0042] When the remote operation function is started, the device connection in progress screen 850 is displayed. The hard key display button 851, when pressed, displays a UI for operating the hard keys 3120 of the MFP110. Details will be described later.

[0043] When the VNC connection is completed and the remote operation function is established, the screen displayed on the DISPLAY 3090 of the MFP 110 is displayed on the operation panel 206. Here, the same screen as the keyboard login screen 910, the keyboard login screen 860, is displayed. Except that the hard key display button 851 is displayed on the operation panel 206, the description is omitted because the screen content is the same as the keyboard login screen 910 of the MFP 110. Also, when the MFP 110 was displaying the ID card login screen 920, the ID card login screen 865 is displayed.

[0044] When a login instruction is given on the keyboard login screen 860, the same menu screen as the menu screen 930 (hard key not displayed) screen 870 is displayed (similarly, the menu screen 930 is also displayed on the DISPLAY 3090).

[0045] When the hard key display button 851 is tapped, the menu display screen (hard key displayed) 875 is displayed. That is, a software keyboard that assigns the processing of each key of the hard key 3120 of the MFP 110 is displayed. The hard key UI 877 is a UI for operating the hard key 3120 of the MFP 110. When a button displayed on the hard key UI 877 is tapped, the pressed key information is sent to the MFP 110, and the MFP 110 performs processing based on the pressed key. When the logout key 879 corresponding to the logout key 901 of the MFP 110 is pressed, logout is performed from the MFP 110. Also, when the end button 878 on the hard key UI 877 is tapped, the remote operation function is terminated, that is, the VNC connection is terminated. When the VNC connection is terminated, the app home screen (device selected) 840 is displayed on the operation panel 206. Also, when the VNC connection is terminated, the screen that was displayed on the DISPLAY 3090 before the VNC connection was terminated continues to be displayed.

[0046] If, without performing an operation to end the VNC connection, a user who has finished a remote operation leaves the MFP application 300 that operates as a VNC client in a non-foreground state (e.g., background), there will be a problem. Devices with a login configuration like an MFP often limit the number of external devices that can be VNC-connected to one in order to limit the number of operators to one. If, despite the remote operation being finished, the user forgets to perform the operation to disconnect the VNC connection, other users will not be able to establish a VNC connection with the MFP. For example, after a user starts the remote operation function and instructs the MFP 110 to perform processes such as printing and scanning from the mobile terminal 100, the user forgets to press the end button 878 and uses another application without pressing the end button 878 (the MFP application 300 transitions to the background). Then, since the VNC connection between the mobile terminal 100 and the MFP 110 continues, other users will not be able to establish a VNC connection with the MFP 110. To solve this problem, when the MFP application 300, which is a VNC client, becomes non-foreground on the mobile terminal 100, control is performed so that other devices can establish a VNC connection to the MFP 110. For example, when a predetermined time elapses from the state where the MFP application 300 becomes non-foreground on the mobile terminal 100, the VNC connection is disconnected or the "priority state" of the VNC connection (details will be described later) is released. In the present embodiment, the state where the application is in the foreground means that the application is in an active state. For example, it is a state where the application is the target of operation on the information processing device. The application is displayed on the information processing device, and the application accepts user operations such as taps and performs corresponding processing. If the application is in the foreground state, the OS of the information processing device preferentially performs the processing of the application. The state where the application is in the background means that the application is operating but not active and is not the target of user operation.

[0047] FIG. 6A is a flowchart showing the processing of the mobile terminal 100 from when it establishes a VNC connection with the MFP 110 and performs remote operations until it disconnects the VNC communication. Each step shown in the flowchart of FIG. 6A is processed by the CPU 201 by expanding and executing a control program stored in a memory such as the ROM 202 in the RAM 203.

[0048] In step S601, when it is detected on the home screen 810 that the icon 811 has been tapped, the MFP application 300 is launched. In step S602, when the screen control unit 301 detects that the device search button 823 has been tapped on the app home screen 820 (device not selected), the MFP search is executed. In step S603, it is determined whether an MFP has been selected on the device search screen 830. If so, the screen is transitioned to the app home screen (device selected) 840, and the process proceeds to step S604. Otherwise, the process returns to S603.

[0049] In step S604, on the app home screen (selected) 840, it is determined whether the remote operation button 822 has been selected. If so, the process proceeds to step S605. Otherwise, the process returns to step S604.

[0050] In step S605, the remote operation unit 313 makes a connection request to the MFP selected in S604 using the RFB protocol. And in S606, remote operation processing is performed.

[0051] The details of the processing in S606 will be described with reference to FIG. 6B. First, in step S607, it is determined whether a connection request using the RFB protocol has been received. If so, the process proceeds to step S609. If not, the process proceeds to step S608. Whether the connection can be accepted is determined by the response data of the connection request. In step S608, the screen control unit 301 displays the connection rejection error screen 880, and the process ends.

[0052] In step S609, an initialization process in the RFB protocol is performed. In the initialization process, the remote operation unit 313 transmits VNC client information and receives VNC server information.

[0053] In step S610, based on the screen information transmitted from the MFP 110, that is, the VNC server, the remote operation unit 313 displays the screen on the operation panel 206. For example, if the screen displayed on the DISPLAY 3090 is the keyboard login screen 910, the keyboard login screen 860 is displayed on the operation panel 206. Also, each time screen information is transmitted from the MFP 110, the screen is sequentially displayed in S610.

[0054] In S611, the remote operation unit 313 determines whether the mobile terminal 100 is in a priority state. If it is in the priority state, the process proceeds to S612; otherwise, it proceeds to S617. Here, the "priority state" means that while the mobile terminal 100 is VNC-connected to the MFP 110, other devices are excluded from being VNC-connected to the MFP 110. For example, for VNC connection, there are a first-preference mode that gives priority to the connection with the device that connected first and controls so that even if another device requests connection later, it cannot be connected, and a later-preference mode that gives priority to the connection with the device that connected later via VNC. If there is a device that requests connection later, the connection with the previously connected device is disconnected and the connection is made with the device that requested connection later. In the case of the first-preference mode, since subsequent connections can be excluded when the VNC connection is successful, it is considered to be in the "priority state" when the connection is successful. On the other hand, in the case of the later-preference mode, even if the VNC connection is successful, the connection will be disconnected if a VNC connection request comes from another device later. Therefore, in the case of the later-preference mode, for example, when a predetermined time has elapsed since the VNC connection, it transitions to the "priority state". During the "priority state", even if the connection is disconnected because another device connects via VNC later, it automatically reconnects to VNC and controls to resume the VNC connection. That is, by performing this control, ostensibly a device that connected via VNC later cannot make a connection. In the case of the later-preference mode, this state is considered the "priority state". Here, an example where it becomes the "priority state" when a predetermined time has elapsed since the VNC connection is shown, but it is not limited to this, and it may be configured to transition to the "priority state" when logging in to the MFP 110 via VNC connection.

[0055] In S612, the remote operation unit 313 determines whether the MFP application 300 has transitioned to the background on the mobile terminal 100. If it is in the background, the process proceeds to S613; otherwise, that is, if it is in the foreground, the process proceeds to S617. When the MFP application 300 transitions to the background, the OS 350 notifies the MFP application 300 that it has transitioned to the background, and the MFP application 300 detects that it has transitioned to the background.

[0056] In S613, the remote operation unit 313 determines whether the VNC connection format is the first-priority format or the second-priority format. You may inquire with the MFP 110 about which format it is. If it is the first-priority format, proceed to S614; if it is the second-priority format, proceed to S615.

[0057] In S614, the remote operation unit 313 releases the priority state, and in S620, it performs the VNC connection disconnection process. In the case of the first-priority format, the VNC connection becomes the "priority state" upon successful connection, so the release of the "priority state" is the same as disconnecting the VNC connection. Here, when the MFP application 300 enters the background state, that is, the processes of S614 and S620 proceed. However, the processes of S614 and S620 may also proceed after a predetermined time has elapsed. That is, the VNC connection may be disconnected immediately when the MFP application 300 enters the background state, or it may be disconnected after a predetermined time has elapsed since it entered the background state.

[0058] In S615, the remote operation unit 313 releases the priority state. Here, releasing the priority state means setting it to a state where even if another device subsequently makes a VNC connection to the MFP 110, it will not reconnect and resume the connection. That is, it transitions from the "priority state" where another device reconnects to the MFP 110 and resumes the connection to a state where another device does not reconnect and does not resume the connection even if it makes a VNC connection to the MFP 110. Thus, another device can make a VNC connection to the MFP 110. Then, proceed to S616.

[0059] In S616, the remote operation unit 313 determines whether the MFP application 300 has returned to the foreground within a certain period. If it has returned, proceed to S617; if not, proceed to S620 and perform the VNC connection disconnection process. The disconnection process is, for example, to send the operation information corresponding to the end button 878 to the MFP 110 and request the MFP 110 to disconnect the VNC.

[0060] In S617, the remote operation unit 313 determines whether there is an input operation on the mobile terminal 100. If there is an input operation, the process proceeds to S618; otherwise, it proceeds to S611. In S618, the remote operation unit 313 transmits the operation information of the operation received on the mobile terminal 100 to the MFP 110.

[0061] In S619, the remote operation unit 313 determines whether it has received an instruction to disconnect the VNC connection. If it has received the instruction, the process proceeds to S620; otherwise, it proceeds to S611. The instruction to disconnect the VNC connection is, for example, the pressing of the end button 878.

[0062] According to the above flowchart, when the mobile terminal 100 establishes a VNC connection with the MFP 110 and the application that operates as a VNC client is placed in the background on the mobile terminal 100 in response to the user finishing the process, even if the user forgets to instruct the disconnection of the VNC connection, other devices can establish a VNC connection with the MFP 110. Also, since the VNC connection is stopped when the condition that a predetermined time has elapsed after the application is placed in the background is satisfied, for example, even when other applications such as a phone call are used during the VNC connection, the VNC connection can be continued within the predetermined time, and the operation of the MFP can also be continued.

[0063] In the above flowchart, an example of the case where the application is in the background state has been described. However, it is not limited to this, and the VNC connection may also be stopped when the application is closed by the user. In this case, instead of stopping after a predetermined time has elapsed, when the application is closed, the mobile terminal transmits a VNC disconnection request to the MFP and immediately disconnects the connection.

[0064] Next, the processing of the MFP 110 when the mobile terminal 100 is screen-linked with VNC using FIG. 7 will be described. Each step shown in the flowchart of FIG. 7 is processed by the CPU 3020 expanding and executing the control program stored in the memory such as the ROM 3030 in the RAM 3040.

[0065] In step S701, the connection determination unit 523 determines whether there is a VNC connection request from the mobile terminal 100 which is a VNC client. If there is a connection request, it proceeds to step S702. If there is no connection request, it returns to step S701. In step S702, the connection determination unit 523 performs negotiation processing to establish a connection with the VNC client. In step S703, the connection determination unit 523 performs initialization processing, receiving client information and transmitting server information.

[0066] In step S704, the screen transmission unit 521 transmits the screen displayed on the DISPLAY 3090 to the mobile terminal 100 which is a VNC client.

[0067] In step S705, the operation information reception unit 522 determines whether it has received an operation from the user. If it has received an operation, it proceeds to step S706. If not, it returns to step S705. For example, when operation information is received from the mobile terminal 100 or a user operation is detected by the PANEL 3070 of the MFP, the hard key 3120, etc., it is determined that an operation has been received from the user.

[0068] In S706, it is determined whether the operation received in S705 is a VNC connection disconnection operation. If it is a disconnection operation, it proceeds to S707 to disconnect the VNC connection. If not, it proceeds to S708.

[0069] In S708, processing corresponding to the operation received in S705 is performed. For example, processing to transition the menu screen displayed on the DISPLAY 3090 to the next page, processing to log in the user to the MFP 110, processing to log out, etc. Next, in S709, the operation information reception unit 522 determines whether the screen displayed on the DISPLAY 3090 has changed according to the processing performed in S708. If it has changed, it proceeds to S704. If not, it proceeds to S705.

[0070] Through the above processing, the screen displayed on the MFP can also be displayed on the mobile terminal. Based on the operations received by the MFP or the mobile terminal, the screens displayed on the MFP and the mobile terminal can be changed. In the above flowchart, an explanation was given of changing the screen displayed on the mobile terminal based on the operations received by the MFP or the mobile terminal. However, even when the screen displayed by the MFP is changed without receiving an operation on the MFP or the mobile terminal, the MFP similarly transmits the changed screen information to the mobile terminal, and the mobile terminal displays the changed screen. Also, through the processing of the above flowchart, the VNC connection can be disconnected in response to a VNC disconnection request from the mobile terminal. When disconnecting the VNC connection, if the user is logged in to the MFP110, an automatic logout process may be performed. Additionally, when disconnecting the VNC connection, the MFP may be made to perform an auto-clear process automatically. The auto-clear process is a process of returning the displayed screen to the screen set as the initial screen.

[0071] According to the above embodiments, even if a certain user has not performed a VNC connection disconnection process and an application operating as a VNC client is in the background state, other users can connect to the VNC. In the above embodiments, the mobile terminal considered its own "priority state" to determine the necessity of the VNC connection and sent a VNC connection disconnection request. Not limited to this, the determination of the necessity of the VNC connection considering the "priority state" of the mobile terminal may be made on the MFP side, and the VNC connection may be disconnected. In this case, the MFP obtains information indicating that the application has been put in the background from the mobile terminal and determines the necessity of disconnection.

[0072] (Other Embodiments) In the above embodiments, an example was shown of displaying on the mobile terminal the same screen as the screen displayed by the MFP in the VNC connection. However, it is not limited to this. For example, even in a case where instead of displaying the same screen as the MFP's screen, an extended screen of the MFP's screen is displayed on the mobile terminal, the above embodiments are applicable.

[0073] Furthermore, the present invention can also be realized by executing the following processes. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or device reads and executes the program. In this case, the computer program and the storage medium storing the computer program constitute the present invention.

Explanation of Reference Numerals

[0074] 100 Mobile Terminal 110 MFP 201 CPU 202 ROM 203 RAM 204 HDD 205 RTC 206 Operation Panel 211 Wireless Communication Unit 300 MFP Application

Claims

1. A program for causing a computer of an information processing apparatus that communicates with an image processing apparatus to execute a control method, the control method comprising: a display step of displaying, on a screen corresponding to a screen to be displayed on a display unit of the image processing apparatus, the screen that is display-controlled by the image processing apparatus; a transmission step of, when a predetermined condition is satisfied in a state where the program has stopped operating as an operation target of a user on the information processing apparatus, transmitting an instruction to stop the display control to the image processing apparatus; wherein, when a predetermined connection setting between the image processing apparatus and the information processing apparatus is a first setting, the predetermined condition is that a predetermined time has elapsed in a state where the program has stopped operating as an operation target of the user on the information processing apparatus; and when the predetermined connection setting is a second setting, the predetermined condition is that the predetermined time has elapsed in a state where the program has stopped operating as an operation target of the user on the information processing apparatus without the program starting to operate as an operation target of the user. A program characterized by the above.

2. The state where the program has stopped operating as an operation target of a user on the information processing apparatus is a state where the program has stopped operating in the foreground on the information processing apparatus. The program according to claim 1, characterized by the above.

3. The state where the program has stopped operating as an operation target of a user on the information processing apparatus is a state where the program is operating in the background on the information processing apparatus. The program according to claim 1 or 2, characterized by the above.

4. The state where the program has stopped operating as an operation target of a user on the information processing apparatus is a state where the operation of the program has stopped. The program according to any one of claims 1 to 3, characterized by the above.

5. The instruction to stop the display control is a disconnection of communication for performing the display control. The program according to any one of claims 1 to 4, characterized by the above.

6. When the predetermined connection setting is the second setting, if the program starts to operate as an operation target of the user before the predetermined time elapses in a state where the program has stopped operating as an operation target of the user on the information processing apparatus, an instruction to stop the display control is not transmitted to the image processing apparatus. ​ ​ The program according to any one of claims 1 to 5, characterized in that...

7. The first setting is a state in which, while a predetermined connection is valid, control is performed so that a connection between the image processing apparatus and another apparatus cannot be established. The second setting is a state in which, while the predetermined connection is valid, control is performed so that a connection between the image processing apparatus and another apparatus can be established. The program according to any one of claims 1 to 6, characterized in that...

8. When the information processing apparatus is in a priority state in which, when a connection is established between the image processing apparatus and another apparatus while the predetermined connection is valid and the predetermined connection setting is the second setting, the predetermined connection is automatically re-executed. The program according to claim 7, characterized in that...

9. When the program does not operate as an operation target of the user on the information processing apparatus and the predetermined connection setting is the second setting, the priority state is released. The program according to claim 8, characterized in that...

10. The program is a program for giving a print instruction or a scan instruction to the image processing apparatus. The program according to any one of claims 1 to 6, characterized in that...

11. The display control is control for displaying, on the display unit of the information processing apparatus, the same screen as the screen displayed on the display unit of the image processing apparatus. The program according to any one of claims 1 to 7, characterized in that...

12. The information processing apparatus and the image processing apparatus perform a VNC (Virtual Network Computing) connection, and the display control is control using VNC. The program according to any one of claims 1 to 8, characterized in that...

13. An information processing system including an information processing apparatus and an image processing apparatus, a display control unit that performs display control so as to display, on the display unit of the information processing apparatus, a screen corresponding to the screen displayed on the display unit of the image processing apparatus; a program that displays the screen controlled by the display control unit, and a stop means that stops the display control by the display control unit when a predetermined condition is satisfied in a state where the program operating on the information processing apparatus does not operate as an operation target of the user on the information processing apparatus; having When a predetermined connection setting between the image processing apparatus and the information processing apparatus is the first setting, the predetermined condition is that a predetermined time has elapsed in a state where the program has stopped operating as an operation target of the user on the information processing apparatus. When the predetermined connection setting is the second setting, the predetermined condition is that the program has stopped operating as an operation target of the user on the information processing apparatus and the predetermined time has elapsed without the program operating as an operation target of the user. An information processing system characterized by the above.

14. When the program satisfies a predetermined condition in a state where it does not operate as an operation target of the user on the information processing apparatus, an instruction for stopping the display control by the display control unit is transmitted from the information processing apparatus to the image processing apparatus. The information processing system according to claim 13, characterized by the above.

15. When the program satisfies a predetermined condition in a state where it does not operate as an operation target of the user on the information processing apparatus, information indicating the state is transmitted from the information processing apparatus to the image processing apparatus, and the image processing apparatus stops the display control by the display control unit based on the transmitted information. The information processing system according to claim 13, characterized by the above.

16. An information processing apparatus that communicates with an image processing apparatus, a display means for displaying a screen corresponding to the screen displayed on the display unit of the image processing apparatus and displaying the screen whose display is controlled by the image processing apparatus; a transmission means for transmitting an instruction to stop the display control to the image processing apparatus when a predetermined condition is satisfied in a state where the program for displaying the display-controlled screen does not operate as an operation target of the user on the information processing apparatus; having When a predetermined connection setting between the image processing apparatus and the information processing apparatus is the first setting, the predetermined condition is that a predetermined time has elapsed in a state where the program has stopped operating as an operation target of the user on the information processing apparatus. When the predetermined connection setting is the second setting, the predetermined condition is that the program has stopped operating as an operation target of the user on the information processing apparatus and the predetermined time has elapsed without the program operating as an operation target of the user. An information processing apparatus characterized by the above.

17. A control method executed by an information processing apparatus that communicates with an image processing apparatus, a display step of displaying a screen corresponding to the screen to be displayed on the display unit of the image processing apparatus, and displaying the screen whose display is controlled by the image processing apparatus; a transmission step of transmitting an instruction to stop the display control to the image processing apparatus when a predetermined condition is satisfied in a state where the program for displaying the display-controlled screen does not operate as an operation target of the user on the information processing apparatus; characterized in that it has: when a predetermined connection setting between the image processing apparatus and the information processing apparatus is a first setting, the predetermined condition is that a predetermined time has elapsed in a state where the program does not operate as an operation target of the user on the information processing apparatus, and when the predetermined connection setting is a second setting, the predetermined condition is that the program has not operated as an operation target of the user on the information processing apparatus and the predetermined time has elapsed without the program operating as an operation target of the user A control method characterized by the above.

Images