Information processing system, information processing device, and control method and program thereof

Abstract

To solve the problem in which: there is a function called a remote operation for remotely operating an information processing apparatus by using virtual network computing (VNC); an information processing apparatus to be a VNC server may restrict the number of clients to establish VNC connection to one; when an information processing apparatus is newly connected to a second client while being in VNC connection with a first client, the apparatus regularly controls to make effective the connection with the first client connected earlier (priority to former) or make effective the connection with the second client connected later (priority to latter).SOLUTION: One information processing apparatus provides both priority to the former and priority to the latter.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to control when an apparatus is operated remotely. [Background technology]

[0002] There is a function called remote operation that allows you to remotely control a device. This is achieved using software such as VNC (Virtual Network Computing). VNC server software is started on the server side, and VNC client software is started on the client side to connect to the server and perform remote control. The RFB (Remote FrameBuffer) protocol (RFC6143) is used for communication between the VNC server and VNC client. For example, a device with VNC server functionality sends screen information of the operation screen to a terminal with VNC client functionality. Based on the received screen information, the terminal displays the same screen as the operation screen displayed on the device's operation panel. By performing operations on the screen displayed on the terminal, the user can have the device execute processes without having to operate the device's operation panel.

[0003] In an image forming apparatus or other apparatus with limited CPU and memory resources, the number of VNC clients that can be connected and operated simultaneously may be limited to one. In this case, it is necessary to consider the priority order of connections when a second VNC client connects. Such an apparatus either fixedly prioritizes a connection with a client that has connected earlier via VNC (first priority) or fixedly prioritizes a connection with a client that has connected later via VNC (last priority). In the case of first priority, the connection with the client that has connected earlier via VNC is maintained, and a connection with a client that has connected later via VNC is not made. In the case of last priority, the connection with the client that has connected earlier via VNC is disconnected, and a connection is made with a client that has connected later via VNC. Patent Document 1 discloses a method of prioritizing a connection with a client that has connected later via VNC, and making a VNC connection when the client satisfies the conditions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-224114 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, if a VNC server device operates in a fixed first-priority or last-priority manner as in the past, usability decreases. For example, in the case of first-priority, if a client that connected via VNC first forgets to disconnect after finishing remote operation, a client that connects via VNC later will not be able to connect via VNC. Also, in the case of last-priority, if a user is in front of a device and has made a VNC connection to operate it remotely, and later a different user makes a VNC connection remotely, the latter user will take the VNC connection even though he or she is in front of the device.

[0006] In this way, depending on the remote operation situation, the type of client connecting via VNC, the operation of the device, etc., there are cases where first priority is more appropriate and cases where last priority is more appropriate.

[0007] In view of the above, an object of the present invention is to provide a VNC connection method suitable for limiting the number of VNC clients that can be remotely controlled to one. [Means for solving the problem]

[0008] The information processing device of the present invention comprises: An information processing device that operates as a VNC (Virtual Network Computing) server, receiving means for receiving RFB protocol version information from a VNC client; VNC connection with the first VNC client In When a VNC connection request is received from a second VNC client, The version information received by the receiving means Based on The first VNC A first connection control that maintains a connection with the client and controls not to establish a VNC connection with the second VNC client. and, The first VNC a second connection control for disconnecting the connection with the client and controlling to establish a VNC connection with the second VNC client; and, Do one of the following: Means of implementation and The present invention is characterized by having do. Effect of the Invention

[0009] This provides a VNC connection method that is suitable for limiting the number of VNC clients that can be remotely controlled to one. [Brief description of the drawings]

[0010] [Figure 1] A diagram showing an example of a system configuration. [Diagram 2] Hardware configuration diagram of image forming apparatus 102 [Diagram 3] Software configuration diagram of the image forming apparatus 102 [Figure 4] Hardware configuration diagram of mobile terminal 103 [Diagram 5] Software configuration diagram of mobile terminal 103 [Figure 6] Flowchart explaining the VNC server connection process [Figure 7] Flowchart explaining the VNC client connection process [Figure 8] Screen displayed on the operation panel 406 of the mobile terminal 103 [Figure 9] 11 is a flowchart for explaining a VNC server connection process in the second embodiment. [Figure 10] Setting screen for setting connection priority DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The best mode for carrying out the present invention will be described below with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. In addition, the following embodiments will be described using an image forming apparatus as an example of an information processing apparatus, but the present invention is not limited to this.

[0012] Example 1 FIG. 1 is a diagram showing an example of an information processing system configuration according to an embodiment of the present invention. An image forming apparatus 102 having a VNC (Virtual Network Computing) server function, a personal computer (PC) 101 which is an information processing apparatus having a VNC client function, and an access point 104 are connected to a LAN (Local Area Network) 105. A mobile terminal 103 which is an information processing apparatus having a VNC client function is connected to the LAN 105 via the access point 104. If the image forming apparatus 102 and the mobile terminal 103 have a wireless direct communication function, they can communicate directly without going through the LAN 105. In this embodiment, an example will be described in which the mobile terminal 103 having a VNC client function connects to the image forming apparatus 102 having a VNC server function via VNC. However, the VNC client to be connected may be another device (for example, the PC 101 or another image forming apparatus not shown) as long as it has a VNC client function.

[0013] In the following description, the image forming apparatus 102 operating as a VNC server is referred to as a server, and information processing apparatuses such as the PC 101 and mobile terminal 103 that connect to this server via VNC are referred to as clients.

[0014] Image forming apparatus 102, which has a VNC server function, transmits screen information corresponding to a screen displayed on the display unit of its own apparatus to a client. The client, having acquired the screen information, displays a screen corresponding to the screen displayed on image forming apparatus 102. When an operation is performed on the screen displayed on the client, the operation information is transmitted to image forming apparatus 102, and image forming apparatus 102 performs processing according to the operation. In other words, a client can remotely operate image forming apparatus 102 by establishing a VNC connection to image forming apparatus 102. Note that the server in this embodiment limits the number of clients that can be connected and operated simultaneously to one.

[0015] First, the image forming apparatus 102 starts the VNC server software. Then, it waits for a connection request from a client. Next, the mobile terminal 103 starts the VNC client software. Then, it connects to a desired server by user operation or the like. Communication between the server and the client is performed according to the RFB (Remote Frame Buffer) protocol. The RFB protocol first performs a handshake process. After the handshake process is completed, screen information and operation information are transmitted and received. In this embodiment, when the handshake process is executed, the client transmits information specifying the priority order of the connection. The server determines the priority order of the connection according to the received information. In other words, if it is specified that the connection with the client that connected first via VNC is to be prioritized (hereinafter referred to as "first priority") and if another client is already connected, the client that connects later via VNC cannot make a VNC connection, or the later VNC connection is disconnected. Also, if it is specified that the connection with the client that connected later via VNC is to be prioritized (hereinafter referred to as "last priority") and if another client is already connected, the connection with the client that connected earlier via VNC is disconnected. In this way, the image forming apparatus 102 is configured to be capable of operating in both a first-priority and a last-priority mode.

[0016] 2 is a diagram showing the hardware configuration of the image forming apparatus 102. In a controller unit 201 in the image forming apparatus 102, a CPU 202 is a computing device that controls the entire system. The CPU 202 comprehensively controls each device connected to a system bus 212.

[0017] The ROM 204 is a non-volatile memory, and stores image data and other data, various programs for the CPU 202 to operate, and the like.

[0018] The RAM 203 is a volatile memory, and is used as a temporary storage area such as a main memory or a work area for the CPU 202. The RAM 203 also stores programs and data such as an operating system, system software, and application software.

[0019] The CPU 202 loads a program stored in the ROM 204 or the like into the RAM 203 and executes it to control each unit of the image forming apparatus 102. Note that the program for the CPU 202 to operate is not limited to the one stored in the ROM 204, and may be one stored in advance in the hard disk drive (HDD) 205 or the like.

[0020] The hard disk drive (HDD) 205 stores the operating system, system software, application software, image data, setting data, etc. However, other storage devices such as SSD, SD memory card, or eMMC may be used as long as they are capable of storing data.

[0021] The display / operation unit I / F 206 is an interface unit with the display / operation unit 207, and outputs information to be displayed on the display / operation unit 207 to the display / operation unit 207. It also accepts information input by the user from the display / operation unit 207. The display / operation unit 207 includes a touch panel and hard keys for accepting user operations. The touch panel is a panel configured in a plane, and notifies the display / operation unit I / F 206 of coordinate information touched by the user. The hard keys are physically depressible keys such as numeric keys, start keys, and stop keys, and notify the display / operation unit I / F 206 when a key is pressed. The display / operation unit 207 also includes a display for displaying images. The display receives a GUI (Graphical User Interface) screen generated by the CPU 202 from the display / operation unit I / F 206 and displays it. By arranging the touch panel integrally on the upper layer of the display, it becomes possible for the user to control the screen displayed on the display as if he or she were directly operating it.

[0022] The network I / F 208 is connected to a network (LAN) 105, and communicates with the PC 101 and other image forming apparatuses (not shown) connected to the same LAN, and inputs and outputs information to and from the mobile terminal 103 via the access point 104.

[0023] A device I / F 209 connects a scanner 210 and a printer 211, which are image input / output devices, to the controller unit 201, and inputs and outputs image data, that is, prints image data on paper, scans documents, and the like.

[0024] 3 is a diagram showing the software configuration of the image forming apparatus 102. System software 301 is stored in any one of the storage means, the RAM 203, the ROM 204, and the HDD 205, and is executed by the CPU 202.

[0025] A job control processing unit 302 controls each software module including a software module not shown, and controls jobs executed by the image forming apparatus 102, such as copying, printing, scanning, and the like.

[0026] The scan processing unit 303 controls the scanner 210 via the device I / F 209 based on an instruction from the job control processing unit 302 , and performs processing to read a document placed on the document platen of the scanner 210 .

[0027] A print processing unit 304 controls a printer 211 via a device I / F 209 based on an instruction from a job control processing unit 302, and performs printing of a specified image.

[0028] The operation input analysis unit 305 analyzes information supplied from the display / operation unit 207 via the display / operation unit I / F 206, and notifies an event corresponding to a user operation to the event processing unit 307. The information analyzed by the operation input analysis unit 305 is operation information indicating that the user has touched the touch panel or pressed a hard key.

[0029] An event analysis unit 306 receives and analyzes various events occurring in the image forming apparatus 102 other than operation input from the job control processing unit 302 or the like, and notifies an event corresponding to the event to an event processing unit 307. The various events analyzed by the event analysis unit 306 are events other than those occurring due to user operations, such as the end of a job or a shortage of printing paper.

[0030] The event processing unit 307 receives an event from the operation input analysis unit 305, the event analysis unit 306, the operation information receiving unit 322, or the like. Depending on the event, the event processing unit 307 notifies the job control processing unit 302, the screen generation unit 308, or the soft keyboard control unit 310 of the event, and controls each unit of the image forming apparatus 102.

[0031] A screen generation unit 308 performs processing for generating, on the RAM 203, data for a screen to be displayed on the display / operation unit 207 or a display unit of an external device.

[0032] The screen display unit 309 reads the screen data generated by the screen generation unit 308 from the RAM 203 , and performs display control to display the data on the display of the display / operation unit 207 via the display / operation unit I / F 206 .

[0033] The soft keyboard control unit 310 performs control to display a soft keyboard screen upon receiving a notification from the event processing unit 307. The soft keyboard control unit 310 also holds information indicating whether or not a soft keyboard is being displayed on the display.

[0034] The VNC server 320 is a software module for communicating with a client via the LAN 105. For example, the VNC server 320 communicates with a VNC client 520 operating on the mobile terminal 103 and performs processing for accepting remote operations.

[0035] The connection processing unit 321 receives a connection request from, for example, a connection processing unit 521 of a VNC client 520, and performs connection control.

[0036] The operation information receiving unit 322 receives operation information from, for example, the operation information transmitting unit 522 of the VNC client 520, and notifies the event corresponding to the user operation to the event processing unit 307. Note that the event notified by the operation information receiving unit 322 is equivalent to the event notified by the operation input analysis unit 305, and is an event for the user to operate the image forming apparatus 102.

[0037] The screen transmission unit 323 reads the screen data generated by the screen generation unit 308 from the RAM 203 and transmits it to the screen reception unit 523 of the VNC client 520 .

[0038] 4 is a diagram showing the hardware configuration of the mobile terminal 103. In the mobile terminal 103, a CPU 401 is a computing device that controls the entire system. The CPU 401 controls each device connected to a system bus 411 in an overall manner.

[0039] The ROM 402 is a non-volatile memory, and stores image data and other data, various programs for the CPU 401 to operate, and the like.

[0040] The RAM 403 is a volatile memory, and is used as a temporary storage area such as the main memory and work area of ​​the CPU 401. The RAM 403 also stores programs and data such as an operating system, system software, and application software.

[0041] The CPU 401 loads a program stored in the ROM 402 or the like into the RAM 403 and executes the program to control each unit of the mobile terminal 103. Note that the program for the operation of the CPU 401 is not limited to the one stored in the ROM 402, and may be one stored in advance in the storage 404 or the like.

[0042] The storage 404 stores the operating system, system software, application software, photos, electronic documents, setting data, etc. However, other storage devices such as SSDs, SD memory cards, and eMMCs may be used as long as they are capable of storing data.

[0043] The RTC (Real Time Clock) 405 keeps track of time.

[0044] The operation panel 406 has a touch panel function capable of detecting a touch operation by a user, and displays various screens provided by the OS 530 and the MFP application 502. The user can input a desired operation instruction to the mobile terminal 103 by inputting a touch operation on the operation panel 406.

[0045] The speaker 407 and microphone 408 are used, for example, when the user makes a call to another mobile terminal or a landline phone.

[0046] The camera 409 captures images in response to an image capture instruction from the user. The photos captured by the camera 409 are stored in a predetermined area of ​​the storage 404.

[0047] The wireless communication unit 410 executes wireless communication such as a wireless LAN.

[0048] 5 is a diagram showing the software configuration of the mobile terminal 103. System software 501 is stored in any one of the storage means, RAM 403, ROM 402, and storage 404, and is executed by the CPU 401.

[0049] The OS 530 is software for controlling the overall operation of the mobile terminal 103. Various applications can be installed in the mobile terminal 103, including an MFP application 502 described later. The OS 530 exchanges information with these applications, and changes the screen displayed on the operation panel 406 according to instructions received from the applications. The OS 530 also includes a group of device drivers for controlling various hardware, and provides an API (Application Programming Interface) for utilizing various hardware to applications running on the OS 530. In this embodiment, a wireless LAN control unit 531 exists as one of the device driver group.

[0050] The wireless LAN control unit 531 is a device driver for controlling the wireless communication unit 410 .

[0051] The MFP application 502 is an application installed in the mobile terminal 103. Using the MFP application 502, it is possible to give print or scan instructions to an image forming apparatus such as the image forming apparatus 102, remotely control the operation screen of the image forming apparatus, and the like. In addition to this MFP application 502, various other applications are installed in the mobile terminal 103, but a description thereof will be omitted.

[0052] The software configuration of the MFP application 502 will now be described in more detail.

[0053] The screen control unit 503 controls the screen to be displayed on the operation panel 406 via the OS 530. The screen control unit 503 displays an MFP application screen on the operation panel 406. The screen control unit 503 also determines an operation instruction input by the user via the operation panel 406. The screen control unit 503 also reads out image data that the screen reception unit 523 has recorded in the application storage unit 506, and displays it on the operation panel 406 via the OS 530.

[0054] The communication unit 504 controls wireless communication by the wireless communication unit 410 via the OS 530 to transmit and receive commands.

[0055] The timer unit 505 measures time.

[0056] An application storage unit 506 temporarily stores various information for executing the MFP application 502 in the RAM 403 .

[0057] The device search control unit 507 generates search data for searching for a connected image forming device 102, transmits the search data to a device on the LAN 105 by the wireless communication unit 410, and receives a response from the corresponding image forming device 102. The search data may be broadcast to all devices on the LAN 105, or may be transmitted by specifying the address of one device.

[0058] The print job generating unit 508 generates a print job. The print job generated by the print job generating unit 508 is transmitted to the image forming apparatus 102 by the wireless communication unit 410, and printing is performed by the image forming apparatus 102.

[0059] The scan job control unit 509 issues a scan instruction to the image forming apparatus 102 via the wireless communication unit 410, and displays the scan data received from the image forming apparatus 102. When the scan data is to be saved, it is stored in the application storage unit 506.

[0060] The VNC client 520 is a software module for remotely controlling the image forming apparatus 102 by communicating with the VNC server 320 running on the image forming apparatus 102 via the LAN 105 .

[0061] The connection processing unit 521 receives a request from a user, transmits a connection request to the connection processing unit 321 of the VNC server 320, and performs connection processing.

[0062] An operation information transmission unit 522 receives an operation instruction input by the user via the operation panel 406 from the screen control unit 503 , and transmits it to the operation information reception unit 322 of the VNC server 320 .

[0063] The screen receiving unit 523 receives screen data from the screen transmitting unit 323 of the VNC server 320 and records it in the application storage unit 506 .

[0064] The communication between the VNC server 320 and the VNC client 520 is realized by the CPU 202 and the CPU 401 controlling the network I / F 208 and the wireless communication unit 410, respectively.

[0065] Through the processing of each software module as described above, the same screen as that displayed on the display of the display / operation unit 207 of the image forming apparatus 102 is also displayed on the operation panel 406 of the mobile terminal 103. Then, the user can operate the operation panel 406 of the mobile terminal 103 to cause the image forming apparatus 102 to execute various processes remotely.

[0066] Next, the details of the handshake process based on the RFB protocol will be described. The handshake process is performed by the connection processing unit 321 of the image forming process 102 and the connection processing unit 521 of the mobile terminal 103. The communication processes of the connection processing units 321 and 521 are realized by the CPU 202 and the CPU 401 controlling the network I / F 208 and the wireless communication unit 410, respectively.

[0067] First, the process of the image forming apparatus 102, which is a VNC server, will be described.

[0068] 6 is a flowchart for explaining the connection process of the image forming process 102. The connection process is realized by the CPU 202 reading out the programs for implementing each control module stored in the ROM 204 or HDD 205 into the RAM 203 and executing them.

[0069] When the user turns on the power of the image forming processing device 102, the VNC server is started by a system in the image forming processing device 102. In the VNC server start-up process, the connection processing unit 321 opens a port for VNC connection and waits for a connection from a client (S601).

[0070] The user starts the remote operation function by operating the mobile terminal 103, and the mobile terminal 103 requests a VNC connection to the image forming process 102. The connection processing unit 321 determines whether or not a connection request has been received from a client (S602), and if a request has been received, the process proceeds to S603, and if not, the process of S602 is repeated. When a connection request is received from a client, the connection processing unit 321 transmits protocol version information to the client (S603). The protocol version transmitted by the server is the version number of the RFB protocol supported by the server. For example, in the case of a server supporting RFB protocol 3.3, "RFB 003.003\n" is transmitted.

[0071] After transmitting the protocol version to the client, the connection processing unit 321 judges whether or not protocol version information has been received from the client (S604). If the protocol version has been received, the process proceeds to S605, otherwise the process of S604 is repeated. Here, the protocol version received from the client is the version number of the RFB protocol actually used in communication between the server and the client. For example, when using RFB protocol 3.3, "RFB 003.003\n" is received.

[0072] When the connection processing unit 321 receives the protocol version information from the client, it analyzes the received protocol version information (S605).

[0073] The image forming apparatus 102 operates by interpreting a part of the protocol version information received from the client as information for specifying the priority order of the VNC connection. Specifically, the first byte of the minor version of the protocol version received from the client is interpreted as information for specifying the priority order of the VNC connection. In this embodiment, if the first byte is 0, the latter priority is given, and if it is 1, the former priority is given. In other words, if the protocol version received from the client is "RFB 003.003\n", the latter priority is given, and if it is "RFB 003.103\n", the latter priority is given. In this embodiment, an example in which the first byte of the minor version is 0 or 1 to switch the priority order of the connection has been described, but this is not limited to this. For example, the first byte of the minor version may be used to switch between multiple functions. Specifically, the possible values ​​of the first byte of the minor version are 0 to F, and this is regarded as a hexadecimal value. This hexadecimal value is 4 bits in binary. A function is assigned to each of these bits. For example, the lowest bit may be used as the priority of the connection, and the remaining second, third, and fourth bits may be assigned to different functions. Then, by taking the logical sum of each bit, it becomes possible to specify four functions at the same time. For example, when specifying the latter priority in the priority of the connection using this method, the minor version part may take "003", "203", "403", "603", "803", "A03", "C03", or "E03". When specifying the earlier priority in the priority of the connection using this method, the minor version part may take "103", "303", "503", "703", "903", "B03", "D03", or "F03". In addition, it is not limited to using the first byte of the minor version, and for example, the second byte from the beginning of the minor version may be used. In addition, in this embodiment, the protocol version received from the client is used, but this may be another client message of the RFB protocol. For example, the client initialization message, ClientCutText, a new message of the RFB protocol extended in the future, or other unique messages not defined in the RFB protocol may be used.In the case of last priority, a specific value may be specified, and in the case of first priority, nothing may be specified, that is, it may be left blank.

[0074] Based on the result of the analysis in S605, the connection processing unit 321 judges whether the priority of the VNC connection is first-come, first-served (S606). If it is first-come, the connection processing unit 321 judges whether there is a client that has already connected via VNC (S607). Here, it is judged that there is a client that has already connected if a VNC connection from another client has already been received, the handshake process has been completed, and it is in a state where information such as a display screen can be sent and received.

[0075] If it is determined in S607 that there is no previously connected client, the handshake process is terminated and a VNC connection is established with the client that received the connection request in S602 (S610). On the other hand, if it is determined in S607 that there is a previously connected client, the connection processing unit 321 transmits error information indicating that there was a previous VNC connection to the client (S608). This error information may extend the security type transmitted by the server, and may transmit, for example, 3 as the security type. Thereafter, the connection processing unit 321 disconnects the connection corresponding to the connection request received in S602 (S609). In other words, a VNC connection is not established with the client that subsequently connected via VNC, and the connection with the client that previously connected via VNC is maintained.

[0076] If the result of the determination in S606 is that the priority of the VNC connection was not first-priority, that is, it was last-priority, the connection processing unit 321 determines whether or not there is a client that has already been connected via VNC (S611). This determination is the same as in S607, so a description thereof will be omitted. If the result of the determination in S611 is that there is no client that has already been connected, the handshake process is ended and a VNC connection is established with the client that received the connection request in S602 (S610).

[0077] If the result of the determination in S611 is that there is a client previously connected via VNC, the connection processing unit 321 disconnects the connection with the client previously connected via VNC (S612). Then, the handshake process ends, and a VNC connection is established with the client that received the connection request in S602, that is, the client that connected via VNC later (S610). Here, when disconnecting the connection with the client previously connected via VNC, error information may be sent to the client previously connected via VNC. The client that receives the error information then displays information indicating the error.

[0078] If a password is set for establishing a VNC connection, it is necessary to execute a process of receiving information based on the password entered by the user from the client and verifying the password in S610. However, since this process is a known process based on the RFB protocol, a description thereof will be omitted. After the connection is established in S610, an initialization message is received from the client and an initialization message is transmitted from the server based on the RFB protocol. The initialization message exchange may be performed before the connection is cut off in S612. After S610, information such as a display screen can be transmitted and received between the server and the client based on the RFB protocol. Specifically, the screen information transmitted by the screen transmission unit 323 is received by the screen reception unit 523 of the VNC client 520. This allows the same screen as that displayed on the display of the image processing device 102 to be displayed on the display of the mobile terminal 103. Furthermore, the operation information reception unit 322 receives the information transmitted by the operation information transmission unit 522 of the VNC client 520. This allows the image forming device 102 to operate based on the operation accepted on the mobile terminal 103.

[0079] In the above example, the client specifies whether to give first priority or last priority, but the image forming apparatus 102 may determine the type of client and then determine whether to give first priority or last priority. In this case, the image forming apparatus receives a client identifier from the client during VNC connection processing, and determines whether to give first priority or last priority based on the identifier.

[0080] By processing the above flowchart, when the number of clients exchanging screen information via VNC connection is limited to one, the priority order of the VNC connection can be switched between first priority or last priority depending on the client requesting the connection.

[0081] In the case of last priority, even if the user of the client who connected via VNC first has finished remote operation but has forgotten to disconnect the VNC connection, another user can connect via VNC later and perform remote operation. Last priority, which has such an advantage, is preferably provided when the client is a PC that is expected to remotely operate the image forming apparatus 102 from a distant location. This is because, since the PC is distant from the image forming apparatus 102, there is a high possibility that the user will not notice that they have forgotten to disconnect the VNC connection. Therefore, it is recommended that the PC specify last priority when processing the VNC connection.

[0082] On the other hand, in the case of first priority, if a user makes a VNC connection first, the connection will not be taken even if another user makes a VNC connection later. First priority, which has such an advantage, is preferably provided when a mobile terminal that is expected to perform remote operation in front of the image forming apparatus 103 is used as a client. It is assumed that a user who is averse to touching the display of an image forming apparatus shared by multiple users will connect to the image forming apparatus via VNC from a mobile terminal and operate his / her mobile terminal in front of the image forming apparatus to cause the image forming apparatus to perform processing. Therefore, it is better to use first priority for clients (mobile terminals) that are expected to make a VNC connection in front of the image forming apparatus. Therefore, it is recommended that a mobile terminal specify first priority when processing a VNC connection.

[0083] As described above, there are advantages to both first priority and last priority, and the above process has the effect of allowing switching to be performed depending on the client so that the advantages can be maximized.

[0084] Next, a process will be described in which the mobile terminal 103, which is a VNC client, makes a VNC connection.

[0085] 8 shows an example of a screen displayed on the operation panel 406 of the mobile terminal 103. When the mobile terminal 103 is started up, it displays a home screen 810. An MFP application icon 811 is an icon for executing the MFP application 502.

[0086] FIG. 7 is a flowchart illustrating a client connection process executed by the CPU 401 of the mobile terminal 103.

[0087] When it is detected that the user has tapped an MFP application icon 811 on the home screen 810, the OS 530 launches the MFP application 502 (S701). When the MFP application 502 is launched, the screen control unit 503 displays an application home screen (no device selected) 820 on the operation panel 406. A selected device icon 821 displays the currently selected device. If no device is selected, "No Printer" is displayed. A device search button 823 is a button for searching for an image processing device. A remote operation menu 822 is a menu button for using the remote operation function.

[0088] When the screen control unit 503 detects that the device search button 823 has been tapped on the application home screen (device not selected) 820, the device search control unit 507 executes a device search. In addition, the screen control unit 503 displays a device search screen 830 on the operation panel 406 (S702). The device search screen 830 displays a list of searched devices. A search result 831 indicates the image forming apparatus 102, and by selecting this, the image forming apparatus 102 can be selected as the target device. On the device search screen 830, the screen control unit 503 judges whether or not a device has been selected (S703). If the result of the judgment in S703 is that a device has not been selected, the screen control unit 503 returns to S704. If the result of the judgment in S703 is that a device has been selected (for example, it is detected that the search result 831 has been tapped by the user), the screen control unit 503 displays an application home screen (device selected) 840 on the operation panel 406. The device selected on the device search screen 830 is displayed in the selected device icon 821. The devices displayed here are used as destination devices when the MFP application 502 transmits a print job or a scan job, and are also targets of remote operation using the remote operation function.

[0089] On the application home screen (device selected) 840, the screen control unit 503 determines whether or not the remote operation menu 822 has been selected (S704). If the result of the determination in S704 is that the remote operation menu 822 has not been selected, the screen control unit 503 returns to S704. If the result of the determination in S704 is that the remote operation menu 822 has been selected, the MFP application 502 starts the remote operation function. When the remote operation function is started, the screen control unit 503 displays a device connection screen 850 on the operation panel 406. The hard key display button 851 is a button for displaying a hard key screen for operating the hard keys of the image forming apparatus 102 from the mobile terminal 103.

[0090] When the remote operation function is started, the connection processing unit 521 transmits a connection request to the server (here, the image forming apparatus 102) (S705). After transmitting the connection request, the connection processing unit 521 judges whether or not the protocol version has been received from the server (S706). For example, in the case of a server that supports RFB protocol 3.3, "RFB 003.003\n" is received. After receiving the protocol version from the server, the connection processing unit 521 transmits the protocol version (S707), and if not, repeats the process of S706. As described above, the protocol version transmitted by the client is the version number of the RFB protocol actually used in the communication between the server and the client. For example, when using RFB protocol 3.3, "RFB 003.003\n" is transmitted. Part of the protocol version information transmitted by the client can be information specifying the priority order of the VNC connection. Here, for example, if the client is a mobile terminal, the application specifies first priority, and if it is a PC, it specifies last priority. In addition, first priority or last priority may be different depending on the operating system (OS) running on the client or the application for VNC connection. For example, first priority may be specified for the MFP application 502, and last priority may be specified for other VNC applications, or the MFP application 502 may determine the type of terminal or OS on which it is running and specify first priority or last priority. In this way, whether first priority or last priority is specified is different depending on the client.

[0091] As described above, the first byte of the minor version may be used to specify the connection priority, and multiple functions may be switched. Alternatively, the second byte from the beginning of the minor version may be used. Alternatively, a different client message of the RFB protocol may be used.

[0092] After transmitting the protocol version, the connection processing unit 521 judges whether or not error information has been received from the server (S708). This is judged based on the security type received from the server. For example, if the security type received is 3, which is an extended version of the specification, it is judged to be an error indicating that there was a previous client connected via VNC.

[0093] If it is determined in S708 that no error information has been received, the connection processing unit 521 ends the handshake process of the connection request sent in S705, and a VNC connection is established. If it is determined in S708 that error information has been received, the connection processing unit 521 disconnects the connection corresponding to the connection request sent in S705, and the screen control unit 503 displays an error screen 860 on the operation panel 406 (S709). The OK button 861 is a button for closing the error screen 860. When the OK button is pressed, the screen control unit 503 closes the error screen 860 on the operation panel 406, and displays the application home screen (device selected) 840.

[0094] If no error information is received in S708, the handshake process of the connection request sent in S705 ends successfully. After this, an initialization message is sent from the client and an initialization message is received from the server based on the RFB protocol. After that, information such as a display screen is sent and received based on the RFB protocol. For example, if a keyboard login screen is displayed on the display of the display / operation unit 207 of the image forming apparatus 102, the same keyboard login screen 870 can be displayed on the operation panel 406 of the mobile terminal 103. In addition, when a user name and password are entered and a login button 871 is pressed, the entered information is sent to the server, and the image forming apparatus 102 can be remotely operated in the same way as if the display / operation unit 207 of the image forming apparatus 102 was operated. After logging in, it becomes possible to remotely instruct the print function and scan function of the image forming apparatus 102.

[0095] By processing the above flowchart, the mobile terminal can specify whether to connect with first priority or last priority when connecting via VNC. If first priority is specified and there is a client already connected via VNC, an error screen will be displayed.

[0096] In this embodiment, the number of clients connected to the server via VNC is one, and if a second client connects to the server after a first client has already connected via VNC, the connection to either the first or second client is disconnected. However, it is not always necessary to disconnect the VNC connection, as long as the number of clients remotely operating the server can be limited to one. In other words, the VNC connection is made with both the first and second clients, and transmission and reception of screen information, etc. is made only with the first or second client.

[0097] Example 2 In the first embodiment, when client A has already connected via VNC and another client B later requests a VNC connection, whether or not to disconnect the VNC connection with client A is switched depending on whether client B has specified first priority or last priority. In the second embodiment, an example is shown in which whether or not to disconnect the VNC connection with client A is switched depending on whether client A has specified first priority or last priority.

[0098] That is, while the first embodiment takes into consideration the priority setting specified by the client that connects via VNC later, the present embodiment takes into consideration the priority setting specified by the client that connects via VNC earlier. The basic configuration of this embodiment is the same as that of the first embodiment, so only the differences are shown.

[0099] Fig. 9 is a flowchart for explaining a server connection process executed by the CPU 202 of the image forming process 102. Since the basic configuration is the same as that of the flowchart of Fig. 6 in the first embodiment, only the differences will be explained.

[0100] The connection processing unit 321 judges whether or not there is a client previously connected via VNC (S901). If it is judged that there is no client previously connected via VNC, the connection processing unit 321 establishes a VNC connection with the client that received the connection request in S602 (S906). Then, the connection processing unit 321 stores priority information specified by the client that established the VNC connection. Priority information is information indicating the priority order of VNC connections specified by the client, and is information specifying whether last priority or first priority is given, as in the first embodiment.

[0101] If it is determined in S901 that there is a client previously connected via VNC, it is determined whether the priority information specified by the client previously connected via VNC is first priority (S902). If it is first priority, error information is sent to the client that sent the connection request in S602, that is, the client that connected via VNC later (S903), and the connection processing unit 321 disconnects the connection corresponding to the connection request received in S602 (S904).

[0102] If it is determined in S902 that the first priority is not given, that is, that the last priority is given, the VNC connection with the client that was connected first via VNC is disconnected (S905). Then, a VNC connection is established with the client that received the connection request in S602 (S906), and the priority information specified by that client is stored (S907).

[0103] By the process of the above flowchart, it is possible to switch whether or not to connect to a client that will connect via VNC later, in accordance with the priority information specified by the client that has already connected via VNC.

[0104] (Other Examples) In the above embodiment, an example of switching between first priority and last priority according to a VNC client has been shown. The example of switching between first priority and last priority is not limited to this. For example, whether to use first priority or last priority may be set in advance on a setting screen provided by the image forming apparatus 102, such as a priority information setting screen 1000 in FIG. 10. When the user selects 1001, the image forming apparatus 102 operates as first priority thereafter, and when the user selects 1002, it operates as last priority thereafter. The setting accepted on the setting screen may be a common setting for the image forming apparatus 102, or may be set for each user. In addition, when the mobile terminal 103 processes a VNC connection, a priority information designation screen 1010 may be displayed on the display of the mobile terminal 103, and the user may be able to designate whether to use first priority or last priority via the screen.

[0105] In the above embodiment, an example has been shown in which the same screen as that displayed by the image forming device via a VNC connection is displayed on the mobile terminal, but this is not limiting. For example, the above embodiment can also be applied to a case in which, instead of displaying the same screen as that of the image forming device, an expanded screen of the image forming device is displayed on the mobile terminal.

[0106] Furthermore, the present invention can also be realized by executing the following process. That is, software (program) that realizes the functions of the above-mentioned embodiments is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the program. In this case, the computer program and the storage medium storing the computer program constitute the present invention.

Claims

1. An information processing device that operates as a VNC (Virtual Network Computing) server, A receiving means for receiving RFB protocol version information from a VNC client; When a VNC connection request is received from a second VNC client while a VNC connection is being established with a first VNC client, the receiving means performs the following process based on the version information received by the receiving means: a first connection control for controlling to maintain a connection with the first VNC client and not to establish a VNC connection with the second VNC client; a second connection control for controlling to disconnect the connection with the first VNC client and to establish a VNC connection with the second VNC client; and a means for executing one of the following:

13. An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the execution unit switches between executing the first connection control and executing the second connection control based on the version information received from the second VNC client.

3. 2. The information processing apparatus according to claim 1, wherein the execution unit switches between executing the first connection control and executing the second connection control based on the version information received from the first VNC client.

4. 4. The information processing device according to claim 1, wherein the execution means switches between executing the first connection control and executing the second connection control based on the version information received by the receiving means.

5. 5 . The information processing apparatus according to claim 1 , wherein, when a VNC connection with the second VNC client is not to be established in the first connection control, error information is transmitted to the second VNC client.

6. 6. The information processing apparatus according to claim 1, wherein when the connection with the first VNC client is disconnected in the second connection control, error information is transmitted to the first VNC client.

7. 7. The information processing device according to claim 1, wherein when the second VNC client is a PC (personal computer), version information of an RFB protocol indicating that the second connection control should be executed is received.

8. 8. The information processing apparatus according to claim 1, wherein, when the second VNC client is a mobile terminal, version information of an RFB protocol indicating that the first connection control should be executed is received.

9. 9. The information processing apparatus according to claim 1, wherein the VNC-connected state is a state in which screen information can be transmitted.

10. 10. The information processing apparatus according to claim 1, wherein the information processing apparatus performs a VNC connection with only one VNC client.

11. 11. The information processing apparatus according to claim 1, which is an image forming apparatus having at least one of a print function and a scan function.

12. An information processing system including an information processing device that operates as a VNC (Virtual Network Computing) server and a terminal that operates as a VNC client, The terminal includes: a transmission means for transmitting, to the information processing device, version information of an RFB protocol as information for designating whether a priority is given to a connection with a VNC client that has been connected via VNC first or a connection with a VNC client that has been connected via VNC later, when performing a VNC connection process with the information processing device; having The information processing device includes: When a VNC connection request is received from the terminal in a state where a VNC client is already connected via VNC, the following is performed based on the version information transmitted by the transmission means: a first connection control for controlling to maintain a connection with the VNC client previously connected via VNC and not to establish a VNC connection with the terminal; a second connection control for disconnecting the connection with the VNC client previously connected via VNC and controlling to establish a VNC connection with the terminal; and a means for executing one of the following: An information processing system comprising:

13. 13. The information processing system according to claim 12, wherein the version information transmitted by the transmitting means is changed depending on the terminal.

14. A control method executed by an information processing device operating as a VNC (Virtual Network Computing) server, comprising: receiving RFB protocol version information from a VNC client; When a VNC connection request is received from a second VNC client while the first VNC client is in a VNC connection state, the a first connection control for controlling to maintain a connection with the first VNC client and not to establish a VNC connection with the second VNC client; a second connection control for controlling to disconnect the connection with the first VNC client and to establish a VNC connection with the second VNC client; An execution step of executing any one of the following: A control method comprising the steps of:

15. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 11.

Images