Remote operation method, remote operation program, and information processing terminal

Abstract

To make it possible to facilitate preparation for an information processing terminal to receive control from a remote place.SOLUTION: Operation of a further other information processing terminal not connected to the Internet is controlled according to an operation performed on another information processing terminal connected to the Internet. Operation information indicating the operation performed on the other information processing terminal is received via the Internet, and based on the operation information, a command to connect the further other information processing terminal to the Internet by the other information processing terminal is transmitted to the further other information processing terminal by wired communication or wireless communication.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a remote operation method, a remote operation program, and an information processing terminal.

Background Art

[0002] Due to the impact of COVID-19, the opportunities for people who are not familiar with operating information processing terminals and the elderly to perform telework and online meetings have increased. When a problem occurs in an information processing terminal during telework or an online meeting, if a support staff only instructs the operator of the information processing terminal to operate the information processing terminal by phone, the problem occurring in the information processing terminal may not be solved. Therefore, there are cases where the support staff operates the information processing terminal where the problem has occurred from a remote location. When the support staff operates the information processing terminal to be remotely operated from a remote location, generally, the following prior preparations are required. In advance, the operator of the information processing terminal to be remotely operated needs to prepare to connect the information processing terminal to be remotely operated to the Internet or install dedicated software on the information processing terminal to be remotely operated. However, it is difficult for people who are not familiar with operating information processing terminals and the elderly to operate the information processing terminal to be operated and make preparations for the information processing terminal to be remotely operated.

[0003] Patent Document 1 describes a system including a remote operation device installed at a remote location and connected to the Internet, and a device to be remotely operated that is connected to the Internet and operates a mobile phone according to a command supplied from the remote operation device. In the method described in Patent Document 1, a camera connected to the device to be remotely operated captures the display screen of the mobile phone, and the device to be remotely operated transmits the image data of the captured display screen to the remote operation device. In the method described in Patent Document 1, since the remote operation device displays an image on the display device of the remote operation device, the operator of the remote operation device can operate the mobile phone by remote operation while checking the display screen of the mobile phone displayed on the display device of the remote operation device.

Prior Art Documents

Patent Documents

[0004] Patent Document 1 Japanese Patent Application Laid-Open No. 2002-032241 Summary of the Invention Problems to be Solved by the Invention

[0005] In the method described in Patent Document 1, in order for the operator of the remote control device to operate the mobile phone from a remote location by operating the remote control device, it is necessary for the device to be remotely controlled to be pre-connected to the Internet. Thus, since the operator of the device to be remotely controlled needs to pre-connect the device to be remotely controlled to the Internet, in the method described in Patent Document 1, it is not easy to prepare for the information processing terminal to receive control from a remote location.

[0006] An example of the object of the present invention is to provide a remote control method, a remote control program, and an information processing terminal that enable easy preparation for the information processing terminal to receive control from a remote location. Means for Solving the Problems

[0007] In one aspect of the present invention, a remote control method is a method for controlling the operation of still another information processing terminal not connected to the Internet in accordance with an operation made on another information processing terminal connected to the Internet. Further, the remote control method receives operation information indicating an operation made on another information processing terminal via the Internet, and based on the operation information, transmits a command for still another information processing terminal to connect to the Internet by still another information processing terminal to the still another information processing terminal by wired communication or wireless communication.

[0008] In another aspect of the present invention, the remote operation program is a program that controls the operation of still another information processing terminal not connected to the Internet according to an operation performed on another information processing terminal connected to the Internet. The remote operation program causes a computer to have a receiving function of receiving, via the Internet, operation information indicating an operation performed on another information processing terminal, and a transmitting function of transmitting, by wired communication or wireless communication to still another information processing terminal, a command for connecting still another information processing terminal to the Internet based on the operation information.

[0009] In another aspect of the present invention, the information processing terminal controls the operation of still another information processing terminal not connected to the Internet according to an operation performed on another information processing terminal connected to the Internet. The information processing terminal includes a receiving unit that receives, via the Internet, operation information indicating an operation performed on another information processing terminal, and a transmitting unit that transmits, by wired communication or wireless communication to still another information processing terminal, a command for connecting still another information processing terminal to the Internet based on the operation information.

Advantages of the Invention

[0010] The remote operation method, remote operation program, and information processing terminal of the present invention make it possible to easily prepare for the information processing terminal to receive control from a remote location.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Embodiments for Carrying Out the Invention

[0012] [First Embodiment] The first embodiment of the present invention will be described.

[0013] FIG. 1 is a block diagram showing a configuration example of the information processing terminal 1 of the present embodiment. FIG. 2 is a block diagram showing a configuration example of the remote operation system according to the first embodiment of the present invention.

[0014] The information processing terminal 1 of the present embodiment controls the operation of still another information processing terminal 4 not connected to the Internet N according to an operation made on another information processing terminal 2 connected to the Internet N.

[0015] Another information processing terminal 2 includes an input interface unit 21 and an Internet connection unit 22. For example, another information processing terminal 2 is installed at a remote location away from the information processing terminal 1, the smart device 3, and still another information processing terminal 4.

[0016] Before another information processing terminal 2 starts the remote operation of still another information processing terminal 4, the Internet connection unit 22 is previously connected to the Internet N. The input interface unit 21 receives the input of operation information indicating an operation made on another information processing terminal 2. Operation information is input to the input interface unit 21 according to the operation of an input device (not shown) by the user. The input interface unit 21 outputs the operation information to the Internet connection unit 22. For example, the input interface unit 21 includes a mouse as an input device. For example, the input interface unit 21 includes a keyboard as an input device.

[0017] The other information processing terminal 2 may include a display unit (not shown) for displaying an image. For example, the other information processing terminal 2 acquires an image of the display unit (not shown) of the further other information processing terminal 4 in a predetermined manner. Note that the other information processing terminal 2 does not necessarily have to acquire the image of the display unit of the further other information processing terminal 4. Then, the other information processing terminal 2 causes the display unit of the other information processing terminal 2 to display the image of the display unit of the further other information processing terminal 4. The other information processing terminal 2 can input operation information while allowing the user to view the image of the display unit of the further other information processing terminal 4.

[0018] Operation information is input from the input interface unit 21 to the Internet connection unit 22. The Internet connection unit 22 transmits the input operation information to the smart device 3.

[0019] The input interface unit 21 of the other information processing terminal 2 includes a mouse, and operation information when the user operates the mouse will be specifically described. The operation information includes at least one or more of the following information in response to the user's mouse operation. The operation information includes information indicating the coordinates of the mouse cursor, information indicating that a mouse button has been pressed, information indicating that the mouse wheel button has been rotated, and information indicating a mouse drag operation. For example, the information indicating a mouse drag operation may include information indicating that the coordinates of the mouse cursor have changed while the mouse button is pressed, and information indicating the coordinates of the mouse cursor before and after the movement.

[0020] The input interface unit 21 of the other information processing terminal 2 includes a keyboard, and operation information when the user operates the keyboard will be specifically described. The operation information includes information indicating a scan code corresponding to a key input in response to the user's keyboard operation.

[0021] The smart device 3 has a function of connecting to the Internet N and a communication function of communicating with the information processing terminal 1.

[0022] Specifically, the smart device 3 includes a communication unit 31 and an Internet connection unit 32. Before another information processing terminal 2 starts the remote operation of yet another information processing terminal 4, the communication unit 31 of the smart device 3 is pre-connected to the receiving unit 11 of the information processing terminal 1 in a predetermined manner. Also, before another information processing terminal 2 starts the remote operation of yet another information processing terminal 4, the Internet connection unit 32 is pre-connected to the Internet N. The smart device 3 is, for example, a smartphone.

[0023] Yet another information processing terminal 4 is not connected to the Internet. Yet another information processing terminal 4 includes a communication unit 41. The communication unit 41 communicates with the information processing terminal 1 by wired communication or wireless communication.

[0024] With reference to FIGS. 1 and 2, the configuration of the information processing terminal 1 according to the present embodiment will be described in detail.

[0025] The information processing terminal 1 includes a receiving unit 11 and a transmitting unit 12. The receiving unit 11 is connected to the communication unit 31 of the smart device 3. The transmitting unit 12 is connected to the communication unit 41 of yet another information processing terminal 4.

[0026] The receiving unit 11 receives operation information indicating an operation performed by another information processing terminal 2 via the Internet N. For example, as shown in FIG. 2, the receiving unit 11 receives the operation information transmitted by another information processing terminal 2 from the communication unit 31 of the smart device 3 via the smart device 3 connected to the Internet N. Alternatively, the receiving unit 11 may be connected to the Internet N. When the receiving unit 11 is connected to the Internet N, the receiving unit 11 receives the operation information via the Internet N without going through the smart device 3.

[0027] The transmission unit 12 transmits, by wire communication or wireless communication, to yet another information processing terminal 4, a command for connecting to the Internet N of yet another information processing terminal 4 by yet another information processing terminal 2 based on the operation information. For example, the command for connecting to the Internet N of yet another information processing terminal 4 is operation information for connecting yet another information processing terminal 4 to the Internet N, which is transmitted from yet another information processing terminal 2.

[0028] In this way, the information processing terminal 1 controls the operation of yet another information processing terminal 4 not connected to the Internet N according to the operation made by yet another information processing terminal 2 connected to the Internet N. The information processing terminal 1 receives, via the Internet N, operation information indicating the operation made by yet another information processing terminal 2. The information processing terminal 1 transmits, by wire communication or wireless communication, to yet another information processing terminal 4, a command for connecting to the Internet N of yet another information processing terminal 4 by yet another information processing terminal 2 based on the operation information. Therefore, by transmitting operation information to the information processing terminal 1 via the Internet N by yet another information processing terminal 2, the operation of yet another information processing terminal 4 not connected to the Internet N can be controlled. Thereby, the operation of connecting yet another information processing terminal 4 not connected to the Internet N to the Internet N can be performed by operating yet another information processing terminal 2. Also, by connecting the information processing terminal 1 and yet another information processing terminal 4 by wire communication or wireless communication, the preparation for yet another information processing terminal 4 to receive control from a remote location is completed. In this way, the preparation for yet another information processing terminal 4 to receive control from a remote location can be facilitated.

[0029] Next, with reference to FIG. 3, an operation example of the information processing terminal 1 of the present embodiment will be described. FIG. 3 is a flowchart showing an operation example of the information processing terminal 1.

[0030] The reception unit 11 receives, via the Internet N, operation information indicating the operation made by yet another information processing terminal 2 (step S101).

[0031] The transmitting unit 12 transmits, based on the operation information, a command for connecting to the Internet N of still another information processing terminal 4 by still another information processing terminal 2 to the still another information processing terminal 4 by wired communication or wireless communication (step S102).

[0032] Therefore, by the other information processing terminal 2 transmitting the operation information to the information processing terminal 1 via the Internet N, the operation of still another information processing terminal 4 not connected to the Internet N can be controlled. As a result, the operation of connecting still another information processing terminal 4 not connected to the Internet N to the Internet N can be performed by operating the other information processing terminal 2. Further, by connecting the information processing terminal 1 and still another information processing terminal 4 by wired communication or wireless communication, preparations for still another information processing terminal 4 to receive control from a remote location are completed. In this way, preparations for still another information processing terminal 4 to receive control from a remote location can be facilitated.

[0033] [Second Embodiment] Next, the remote operation system according to the second embodiment of the present invention will be specifically described. The second embodiment is different from the remote operation system of the first embodiment in that the transmitting unit 52 of the information processing terminal 5 includes a USB (Universal Serial Bus) terminal, and the communication unit 81 of still another information processing terminal 8 includes a USB port. Further, the second embodiment is different from the remote operation system of the first embodiment in that the other information processing terminal 6 includes a display control unit 63 that causes an image captured by the smart device 7 to be displayed on the display unit 64 of the other information processing terminal 6.

[0034] First, FIG. 4 shows a configuration example of the remote operation system of the present embodiment. The remote operation system of the present embodiment includes an information processing terminal 5, another information processing terminal 6, a smart device 7, and still another information processing terminal 8. The information processing terminal 5 is connected to still another information processing terminal 8 and the smart device 7. Further, the other information processing terminal 6 is connected to the Internet N. The smart device 7 is connected to the information processing terminal 5 and the Internet N.

[0035] In this embodiment, an OCP (Operational Computer Program) 1, which is a program for realizing the functions of each component of the information processing terminal 5, is installed in the information processing terminal 5 in advance. Also, an OCP 3, which is a program for realizing the functions of each component of another information processing terminal 6, is installed in the other information processing terminal 6 in advance. Regarding the installation of the OCP 2, which is a program for realizing the functions of each component of the smart device 7, into the smart device 7, it will be described later. Note that the smart device 7 may have the functions of the information processing terminal 5 described later. For example, an OCP 1, which is a program for realizing the functions of each component of the information processing terminal 5, may be installed in the smart device 7 in the same manner as the OCP 2. When the OCP 1 and the OCP 2 are installed in the smart device 7, the smart device 7 has the functions of the information processing terminal 5. When the OCP 1 and the OCP 2 are installed in the smart device 7, it is the smart device 7, rather than the information processing terminal 5, that controls the operation of another information processing terminal 8.

[0036] With reference to FIG. 4, the configuration of another information processing terminal 6 of this embodiment will be described in detail. Another information processing terminal 6 includes an input interface unit 61 and an Internet connection unit 62. The Internet connection unit 62 is connected to the Internet N, the input interface unit 61, and a display control unit 63. The display control unit 63 is connected to the Internet connection unit 62 and a display unit 64.

[0037] A program called OCP 3 is stored in advance in a storage unit (not shown) of another information processing terminal 6. Another information processing terminal 6 executes processing according to the OCP 3. By another information processing terminal 6 executing processing according to the OCP 3, the functions of the input interface unit 61, the Internet connection unit 62, the display control unit 63, and the display unit 64 described later are realized.

[0038] OCP3 is a program that enables another information processing terminal 6 to implement at least both of the following two functions. OCP3 enables another information processing terminal 6 to implement a function of transmitting operation information generated by tracking the operations of the input devices of another information processing terminal 6 of the user to the smart device 7. OCP3 enables another information processing terminal 6 to implement a function of displaying an image indicated by the image data received from the smart device 7 on the display unit 64.

[0039] The input interface unit 61 receives inputs from the user. Specifically, operation information corresponding to the operation of the input interface unit 61 by the user is input to the input interface unit 61.

[0040] The input interface unit 61 outputs the input operation information to the Internet connection unit 62. Operation information for connecting another information processing terminal 8 to the Internet N is further input to the input interface unit 61 according to the operation of the input interface unit 61 by the user. The operation information to be input is, for example, operation information corresponding to an operation of inputting information indicating a password that needs to be input on a predetermined operation screen in another information processing terminal 8. In addition to the operation information for connecting another information processing terminal 8 to the Internet N, operation information for remotely operating another information processing terminal 8 may be input to the input interface unit 61.

[0041] The input interface unit 61 includes, for example, a mouse which is an input device. Alternatively, the input interface unit 61 includes a keyboard which is an input device. The input interface unit 61 may include, for example, both a mouse and a keyboard, or may include other input devices. When the input interface unit 61 includes two or more types of input devices, the input interface unit 61 may output information indicating the type of the input device and operation information to the Internet connection unit 62 in association with each other. Further, another information processing terminal 8 described later, after receiving information indicating the type of the input device and a command based on the operation information, may perform the process indicated by the command based on the operation information according to the information indicating the type of the input device.

[0042] Specifically, the operation information when the input interface unit 61 includes a mouse and the user operates the mouse will be described. For example, the operation information includes at least one or more of the following information according to the user's operation of the mouse. The operation information includes information indicating the coordinates of the mouse cursor, information indicating that a button of the mouse has been pressed, information indicating that the scroll button of the mouse has been rotated, and information indicating a drag operation of the mouse. For example, the information indicating a drag operation of the mouse may include information indicating that the coordinates of the mouse cursor have changed while the button of the mouse is pressed, and information indicating the coordinates of the mouse cursor before and after the movement.

[0043] Specifically, the operation information when the input interface unit 61 includes a keyboard and the user operates the keyboard will be described. The operation information includes information indicating a scan code corresponding to a key input according to the user's operation of the keyboard.

[0044] Each time the input interface unit 61 receives an input from the user, it outputs the operation information to the Internet connection unit 62. The input interface unit 61 may output an operation log indicating the user's operations during a predetermined time as operation information to the Internet connection unit 62.

[0045] The Internet connection unit 62 is connected to the Internet N. The connection destination of the Internet connection unit 62 included in the Internet N includes the smart device 7.

[0046] The Internet connection unit 62 receives a notification indicating that it is already connected to the information processing terminal 5 from the smart device 7. When receiving a notification indicating that it is already connected to the information processing terminal 5, the Internet connection unit 62 sends a notification requesting installation to the source of the notification indicating that it is already connected to the information processing terminal 5. For example, the Internet connection unit 62 identifies the source based on the IP (Internet Protocol) address included in the notification indicating that it is already connected to the information processing terminal 5.

[0047] Note that an instruction to request installation of a program to the smart device 7 may be input to the input interface unit 61 in response to an operation of an input device (not shown) of the input interface unit 61 by the user. When an instruction to request installation of a program to the smart device 7 is input, the input interface unit 61 outputs a notification requesting installation of a program to the smart device 7 to the Internet connection unit 62. Then, by the Internet connection unit 62 sending a notification requesting installation of a program to the smart device 7, the smart device 7 may be caused to install the program. For example, the smart device 7 that has received a notification requesting installation of a program to the smart device 7 displays a message indicating the storage destination of the program and the installation request on the display unit of the smart device 7. Thereby, the user can easily install the program on the smart device 7.

[0048] The Internet connection unit 62 receives a notification from the smart device 7 indicating that the preparation for remote operation has been completed. When receiving the notification indicating that the preparation for remote operation has been completed, the other information processing terminal 6 further starts the remote operation of the other information processing terminal 8. Also, the other information processing terminal 6 further starts transmitting operation information to the other information processing terminal 8. Note that when receiving the notification indicating that the preparation for remote operation has been completed, the display control unit 63 may cause the display unit 64 to display a message indicating that the preparation for remote operation of the other information processing terminal 8 has been completed.

[0049] When the other information processing terminal 6 further starts the remote operation of the other information processing terminal 8, the Internet connection unit 62 may output a notification indicating the start of the remote operation to the input interface unit 61 and the display control unit 63. When the notification indicating the start of the remote operation is input, the input interface unit 61 may start generating operation information according to the operation of the input interface unit 61 by the user. Also, when the notification indicating the start of the remote operation is input, the display control unit 63 may cause the display unit 64 to display a message indicating the start of the remote operation. Further, the Internet connection unit 62 may request the smart device 7 for image data. When requested for image data, the imaging unit 73 of the smart device 7 may perform imaging, and the Internet connection unit 72 may transmit the image data to the other information processing terminal 6.

[0050] When OCP2 is installed in the smart device 7 according to the operation by the user instead of the control of the other information processing terminal 6, the Internet connection unit 62 receives a notification from the smart device 7 indicating that the preparation for remote operation has been completed. The other information processing terminal 6 may further start the remote operation of the other information processing terminal 8 when receiving both the notification indicating that it is connected and the notification indicating that the preparation for remote operation has been completed.

[0051] Operation information is input to the Internet connection unit 62 from the input interface unit 61. The Internet connection unit 62 transmits the operation information to the smart device 7 via the Internet N.

[0052] In addition, the Internet connection unit 62 receives image data from the smart device 7 via the Internet N. The Internet connection unit 62 outputs the image data to the display control unit 63.

[0053] The image data is input from the Internet connection unit 62 to the display control unit 63. The display control unit 63 causes the display unit 64 to display the image indicated by the input image data.

[0054] The display unit 64 displays the image indicated by the image data. The display unit 64 is under the control of the display control unit 63. The display unit 64 is, for example, a display.

[0055] Next, with reference to FIG. 4, the configuration of the smart device 7 of the present embodiment will be described in detail.

[0056] The smart device 7 includes a communication unit 71 and an Internet connection unit 72. The Internet connection unit 72 is connected to the photographing unit 73.

[0057] Since the configurations of the communication unit 71 and the Internet connection unit 72 of the smart device 7 in the present embodiment are the same as the configurations of the communication unit 31 and the Internet connection unit 32 of the smart device 3 in the first embodiment shown in FIG. 2, the common description will be omitted.

[0058] A program called OCP2 is stored in a storage unit (not shown) included in the smart device 7 by the other information processing terminal 6 or the user. In the present embodiment, when requested by the other information processing terminal 6, the smart device 7 installs OCP2. Further, after installation, the smart device 7 starts OCP2 and executes processing according to OCP2. By the smart device 7 executing processing according to OCP2, the functions of the communication unit 71, the Internet connection unit 72, and the photographing unit 73, which will be described later, are realized.

[0059] OCP2 is a program that enables the smart device 7 to realize the function of transmitting operation information from the smart device 7 to the information processing terminal 1 upon receiving an instruction from another information processing terminal 6 via the Internet N. Further, OCP2 is a program that enables the smart device 7 to realize the function of transmitting image data captured by the imaging unit 73 of the smart device 7 to another information processing terminal 6. OCP2 may be installed on the smart device 7, for example, by a person who receives support from a support staff regarding the operation of another information processing terminal 8. Alternatively, OCP2 may be pre-installed on the smart device 7 by the support staff on behalf of the person who receives support.

[0060] The communication unit 71 is pre-connected to the receiving unit 51 of the information processing terminal 5 by a predetermined method. In this embodiment, the user connects the communication unit 71 and the receiving unit 51 of the information processing terminal 5. When connected to the information processing terminal 5, the communication unit 71 performs registration and authentication of the information processing terminal 5 and performs pairing processing to enable the smart device 7 to communicate with the information processing terminal 5 and call the functions of the information processing terminal 5.

[0061] In this embodiment, first, the user connects the smart device 7 and the information processing terminal 5. Next, the user connects the information processing terminal 5, which will be described later, and yet another information processing terminal 8. The other information processing terminal 6 and the smart device 7 are connected via the Internet N. Note that the order in which the user connects the devices is not limited to the order described above. The user can connect the devices in any order.

[0062] The communication unit 71 receives a notification from the information processing terminal 5 indicating that it is already connected to yet another information processing terminal 8. When receiving a notification indicating that it is already connected to yet another information processing terminal 8, the communication unit 71 outputs a notification indicating that it is already connected to the information processing terminal 5 to the Internet connection unit 72, which will be described later.

[0063] Operation information is input from the Internet connection unit 72 to the communication unit 71. The communication unit 71 transmits the input operation information to the receiving unit 51 of the information processing terminal 5.

[0064] The Internet connection unit 72 is pre-connected to the Internet N. Another information processing terminal 6 is included in the connection destination of the Internet connection unit 72 included in the Internet N.

[0065] A notification indicating that the information processing terminal 5 is already connected is input from the communication unit 71 to the Internet connection unit 72. The Internet connection unit 72 transmits a notification indicating that the information processing terminal 5 is already connected to another information processing terminal 6 via the Internet N.

[0066] The Internet connection unit 72 receives a notification requesting installation from another information processing terminal 6. When receiving a notification requesting installation, the Internet connection unit 72 acquires OCP2 from an external device. Note that the Internet connection unit 62 of another information processing terminal 6 may transmit OCP2 to the smart device 7. The Internet connection unit 72 stores the acquired OCP2 in a storage unit (not shown). When OCP2 is installed, the Internet connection unit 72 transmits a notification indicating that the preparation for remote operation is completed to another information processing terminal 6 via the Internet N.

[0067] Note that even when OCP2 is installed on the smart device 7 by the user, the Internet connection unit 72 transmits a notification indicating that the preparation for remote operation is completed to another information processing terminal 6.

[0068] The Internet connection unit 72 receives operation information from another information processing terminal 6 via the Internet N. The Internet connection unit 72 outputs the operation information to the communication unit 71.

[0069] In addition, image data is input from the imaging unit 73 to the Internet connection unit 72. The Internet connection unit 72 transmits the input image data to another information processing terminal 6 via the Internet N.

[0070] The imaging unit 73 captures an object to be imaged and outputs image data to the Internet connection unit 72. The imaging unit 73 is, for example, a camera built into the smart device 7. The smart device 7 is placed by the user at a position where the imaging unit 73 can further capture the display unit 83 of another information processing terminal 8. For example, after the smart device 7 is connected to each device, the smart device 7 is placed at a position where the imaging unit 73 can further capture the display unit 83 of another information processing terminal 8. The object to be imaged by the imaging unit 73 is the display unit 83 of another information processing terminal 8. The imaging unit 73 continuously or at a predetermined frequency captures the object to be imaged. In addition, the imaging unit 73 outputs image data to the Internet connection unit 72 at a predetermined frequency. Based on the image data of the imaging unit 73, the display unit 64 of another information processing terminal 6 can display an image showing the screen of the display unit 83 of another information processing terminal 8 in real time.

[0071] Note that after the user places the smart device 7 at a position where the imaging unit 73 can further capture the display unit 83 of another information processing terminal 8, the smart device 7 may transmit a notification requesting remote operation to another information processing terminal 6 according to the operation of the user's smart device 7. When the Internet connection unit 62 of another information processing terminal 6 receives a notification requesting remote operation, another information processing terminal 6 may start remote operation.

[0072] The configuration of the information processing terminal 5 of the present embodiment will be described in detail with reference to FIG. 4.

[0073] The configuration of the information processing terminal 5 in the present embodiment is the same as the configuration of the information processing terminal 1 in the first embodiment shown in FIGS. 1 and 2, so the common description is omitted.

[0074] In the storage unit (not shown) of the information processing terminal 5, a program called OCP1 is stored in advance. The information processing terminal 5 executes processing according to OCP1.

[0075] OCP1 is a program that causes the information processing terminal 1 to realize a function of further transmitting operation information to another information processing terminal 8 in response to an instruction received from the smart device 7 that executes processing according to OCP2.

[0076] With reference to FIG. 4, the configuration of the information processing terminal 5 of the present embodiment will be described in detail. The information processing terminal 5 includes a receiving unit 51 and a transmitting unit 52.

[0077] The receiving unit 51 is connected to the communication unit 71 of the smart device 7. The receiving unit 51 receives operation information indicating an operation performed on another information processing terminal 6 via the Internet N. The receiving unit 51 outputs the received operation information to the transmitting unit 52. The connection between the receiving unit 51 and the communication unit 71 of the smart device 7 is made by the user of the remote operation system. In the present embodiment, after the receiving unit 51 is connected to the communication unit 71 of the smart device 7, the transmitting unit 52 and the communication unit 81 of another information processing terminal 8 are further connected.

[0078] A predetermined method can be used for the connection between the receiving unit 51 and the communication unit 71 of the smart device 7. For example, as a method of connecting between the receiving unit 51 and the communication unit 71 of the smart device 7, a connection by Bluetooth (registered trademark), a connection by radio waves, a connection by infrared rays, or a wired connection including a connection by USB can be used, but it is not limited thereto.

[0079] When the information processing terminal 5 and the smart device 7 are connected, each of the information processing terminal 5 and the smart device 7 performs the following processing. Specifically, when the information processing terminal 5 and the smart device 7 are connected, the receiver 511 of the information processing terminal 5 and the communication unit 71 of the smart device 7 perform pairing processing for registering and authenticating each other's devices to enable communication and function calls. Specifically, when connected to the smart device 7, the receiver 51 performs registration and authentication of the smart device 7, and the receiver 51 performs pairing processing to enable communication with the smart device 7 and function calls of the smart device 7. When the transmitter 52 and the communication unit 81 of another information processing terminal 8 are connected, the receiver 51 transmits a notification indicating that it is already connected to another information processing terminal 8 to the smart device 7.

[0080] Operation information is input to the transmitter 52 from the receiver 51. Based on the operation information, the transmitter 52 transmits a command for connecting another information processing terminal 8 to the Internet N by another information processing terminal 6 to the other information processing terminal 8 via wired communication or wireless communication. In this embodiment, the transmitter 52 transmits the command to the other information processing terminal 8 via wired communication. The command may be operation information.

[0081] Alternatively, the transmitter 52 may convert the input operation information into a command corresponding to another information processing terminal 8. When the operation information transmitted from another information processing terminal 6 is not operation information that another information processing terminal 8 can process, the transmitter 52 converts the input operation information into a command corresponding to another information processing terminal 8. The command corresponding to another information processing terminal 8 is, for example, operation information that the information processing terminal 8 can process. Thereby, regardless of whether the information processing terminal 8 can process the operation information transmitted from the information processing terminal 6, the other information processing terminal 6 can control another information processing terminal 8.

[0082] Specifically, the transmission unit 52 is provided with a USB terminal. The transmission unit 52 is connected to another information processing terminal 8 when the USB terminal is inserted into the USB port of the other information processing terminal 8.

[0083] The information processing terminal 5 is recognized by another information processing terminal 8 as a device compliant with HID (Human Interface Device), such as a mouse or keyboard directly connected to the other information processing terminal 8.

[0084] The information processing terminal 5 is, for example, a microcontroller. When the information processing terminal 5 is a microcontroller, the transmission unit 52 is composed of the GPIO (General-Purpose Input / Output) of the microcontroller and a USB terminal having a cable. When the information processing terminal 5 is a microcontroller, the transmission unit 52 is realized by connecting the GPIO of the microcontroller and the end of the cable.

[0085] Note that the information processing terminal 5 may receive a notification indicating that it is connected to the Internet N from another information processing terminal 8. The information processing terminal 5 may notify another information processing terminal 6 of a notification indicating that another information processing terminal 8 is connected to the Internet N via the smart device 7 and the Internet N. When receiving a notification indicating that another information processing terminal 8 is connected to the Internet N, the display control unit 63 of the other information processing terminal 6 may cause the display unit 64 to display a message indicating that another information processing terminal 8 is connected to the Internet N.

[0086] With reference to FIG. 4, the configuration of another information processing terminal 8 of the present embodiment will be described in detail. Another information processing terminal 8 includes a communication unit 81. The communication unit 81 is connected to a display control unit 82. The display control unit 82 is connected to the communication unit 81 and a display unit 83.

[0087] Still another information processing terminal 8 is connected to the information processing terminal 1. Specifically, by inserting the USB terminal of the transmission unit 52 of the information processing terminal 5 into the USB port of the communication unit 81 of still another information processing terminal 8, still another information processing terminal 8 is connected to the information processing terminal 5. Still another information processing terminal 8 is assumed to be an information processing terminal owned by a person receiving support from a support staff at a remote location. Also, when still another information processing terminal 8 is the information processing terminal of the person receiving support, it is assumed that the person receiving support inserts the USB terminal of the transmission unit 52 of the information processing terminal 5 into the USB port of the communication unit 81 of still another information processing terminal 8. Alternatively, when the remote operation system is used for remote work, still another information processing terminal 8 is assumed to be an information processing terminal that a worker operates from a remote location during remote work. When still another information processing terminal 8 is used for remote work, the worker inserts the USB terminal of the transmission unit 52 of the information processing terminal 5 into the USB port of the communication unit 81 of still another information processing terminal 8 in advance before operating another information processing terminal 6 described later.

[0088] Still another information processing terminal 8 may be in a stand-alone state where it functions without being connected to a device other than the information processing terminal 5 or without depending on a device other than the information processing terminal 5. As shown in FIG. 4, still another information processing terminal 8 is not connected to the Internet N.

[0089] The communication unit 81 includes a USB port which is an insertion port for a USB terminal. The communication unit 81 receives a command based on the operation information from the information processing terminal 5. The communication unit 81 outputs the command based on the operation information to the display control unit 82.

[0090] The display control unit 82 controls the image displayed on the display unit 83. Specifically, a command based on the operation information is input to the display control unit 82 from the communication unit 81. The display control unit 82 performs processing according to the command based on the operation information, and causes the display unit 83 to display an image according to the processing result. For example, for connecting to the Internet, a command based on the operation information corresponding to an operation of inputting a password that needs to be input on a predetermined operation screen in another information processing terminal 8 is input to the display control unit 82. Note that the operation of inputting the password is performed on another information processing terminal 6.

[0091] The case where the display control unit 82 performs processing according to the command based on the operation information and causes the display unit 83 to display an image according to the processing result will be specifically described.

[0092] For example, when information indicating the coordinates of the mouse cursor indicating the position of the mouse is included in the command based on the operation information, the display control unit 82 changes the image to be displayed on the display unit 83 to an image in which the mouse cursor is located at the position indicated by the coordinates indicated by the information indicating the coordinates.

[0093] When the command based on the operation information includes information indicating that the button of the mouse has been pressed, the display control unit 82 changes the image to be displayed on the display unit 83 to an image after the button of the mouse has been pressed.

[0094] When the command based on the operation information includes information indicating that the wheel button of the mouse has been rotated or information indicating a drag operation of the mouse, the display control unit 82 performs processing according to the command and causes the display unit 83 to display an image according to the processing result.

[0095] When the command based on the operation information includes information indicating a scan code, the display control unit 82 causes the display unit 83 to display an image after the key corresponding to the scan code has been input.

[0096] When the command based on the operation information includes an operation log indicating the operations of the user during a predetermined time period, the display control unit 82 performs the following operations. The display control unit 82 performs processing according to the command based on the operation information for the operation information indicating the most recent operation among the operation logs, and causes the display unit 83 to display an image according to the processing result. The display control unit 82 performs processing according to the command based on the operation information for the operation information indicating the operation performed next to the most recent operation, and causes the display unit 83 to display an image according to the processing result. In this way, the display control unit 82 performs processing according to the command based on the operation information included in the operation log from the operation information indicating the most recent operation to the operation information indicating the most new operation, and sequentially causes the display unit 83 to display images according to the processing results.

[0097] The display unit 83 displays an image under the control of the display control unit 82. The display unit 83 is, for example, a display.

[0098] In this way, the information processing terminal 5 controls the operation of still another information processing terminal 8 that is not connected to the Internet N according to the operations made by another information processing terminal 6 connected to the Internet N. The information processing terminal 5 receives operation information indicating the operations made by another information processing terminal 6 via the Internet N. The information processing terminal 5 transmits, by wired communication or wireless communication, to still another information processing terminal 8 a command for connecting still another information processing terminal 8 to the Internet N based on the operation information. Therefore, by another information processing terminal 6 transmitting operation information to the information processing terminal 5 via the Internet N, the operation of still another information processing terminal 8 that is not connected to the Internet N can be controlled. Thereby, the operation of connecting still another information processing terminal 8 that is not connected to the Internet N to the Internet N can be performed by operating another information processing terminal 6. Also, by connecting the information processing terminal 5 and still another information processing terminal 8 by wired communication or wireless communication, preparations for still another information processing terminal 8 to receive a control operation from a remote location are completed. In this way, preparations for still another information processing terminal 8 to receive a control operation from a remote location can be facilitated.

[0099] Next, with reference to FIGS. 5 to 13, an operation example of the remote control system according to the present embodiment will be described.

[0100] FIGS. 5 to 7 are sequence diagrams showing operation examples of the remote control system. FIGS. 8 and 9 are flowcharts showing operation examples of the other information processing terminal 6. FIGS. 10 and 11 are flowcharts showing operation examples of the smart device 7. FIG. 12 is a flowchart showing an operation example of the information processing terminal 5. FIG. 13 is a flowchart showing an operation example of still another information processing terminal 8.

[0101] First, with reference to FIG. 5, the operation until the smart device 7 transmits a notification indicating that the preparation for remote control is completed will be described.

[0102] The user connects the receiving unit 51 of the information processing terminal 5 to the communication unit 71 of the smart device 7 (step S201). The receiving unit 51 of the information processing terminal 5 performs pairing processing (step S202). Also, the communication unit 71 of the smart device 7 performs pairing processing (step S203).

[0103] Then, the user inserts the USB terminal of the transmitting unit 52 of the information processing terminal 5 into the USB port of the communication unit 81 of still another information processing terminal 8. When the transmitting unit 52 of the information processing terminal 5 is connected to the communication unit 81 of still another information processing terminal 8 (step S204), the receiving unit 51 of the information processing terminal 5 transmits a notification indicating that it is connected to still another information processing terminal 8 to the smart device 7 (step S205).

[0104] The communication unit 71 of the smart device 7 receives a notification indicating that it is already connected to another information processing terminal 8 from the information processing terminal 5. When receiving a notification indicating that it is already connected to another information processing terminal 8, the communication unit 71 outputs a notification indicating that it is already connected to the information processing terminal 5 to the Internet connection unit 72. The Internet connection unit 72 transmits a notification indicating that it is already connected to the information processing terminal 5 to another information processing terminal 6 via the Internet N (step S206).

[0105] The Internet connection unit 62 of another information processing terminal 6 receives, via the Internet N, a notification indicating that it is already connected to the information processing terminal 5 from the smart device 7. The Internet connection unit 62 transmits, via the Internet N, a notification requesting installation to the source of the notification indicating that it is already connected to the information processing terminal 5 (step S207).

[0106] The Internet connection unit 72 of the smart device 7 receives a notification requesting installation from another information processing terminal 6. When receiving a notification requesting installation, the Internet connection unit 72 acquires OCP2 from an external device. The Internet connection unit 72 installs OCP2 on the smart device 7 by storing the acquired OCP2 in the storage unit (step S208). The Internet connection unit 72 transmits, via the Internet N, a notification indicating that the preparation for remote operation is complete to another information processing terminal 6 (step S209).

[0107] The Internet connection unit 62 of another information processing terminal 6 receives a notification indicating that the preparation for remote operation is complete from the smart device 7.

[0108] Next, with reference to FIG. 6, the operations until another information processing terminal 6 transmits operation information to the smart device 7, and further another information processing terminal 8 performs processing according to a command based on the operation information and displays an image according to the processing result will be described.

[0109] Operation information corresponding to an operation on the input interface unit 61 by the user is input to the input interface unit 61 of the other information processing terminal 6. The input interface unit 61 outputs the input operation information to the Internet connection unit 62.

[0110] The Internet connection unit 62 transmits the operation information to the smart device 7 via the Internet N (step S301).

[0111] The Internet connection unit 72 of the smart device 7 receives the operation information from the other information processing terminal 6 via the Internet N. The communication unit 71 transmits the operation information to the receiving unit 51 of the information processing terminal 5 (step S302).

[0112] The receiving unit 51 of the information processing terminal 5 receives the operation information. The operation information is input from the receiving unit 51 to the transmitting unit 52. Based on the operation information, the transmitting unit 52 transmits, by wire communication, a command for connecting to the Internet of another information processing terminal 8 by another information processing terminal 6 to the other information processing terminal 8 (step S303).

[0113] Furthermore, the communication unit 81 of the other information processing terminal 8 receives a command based on the operation information from the information processing terminal 5. The command based on the operation information is input from the communication unit 81 to the display control unit 82. The display control unit 82 performs processing according to the command based on the operation information (step S304), and causes the display unit 83 to display an image according to the processing result. The display unit 83 displays an image according to the processing result (step S305).

[0114] Next, with reference to FIG. 7, an operation in which the other information processing terminal 6 causes the display unit 64 to display image data indicating an image captured by the smart device 7 will be described.

[0115] The smart device 7 is placed by the user at a position where the imaging unit 73 can image the display unit 83 of another information processing terminal 8.

[0116] The imaging unit 73 of the smart device 7 further images the display unit 83 of another information processing terminal 8 (step S401), and outputs the image data to the Internet connection unit 72. The Internet connection unit 72 transmits the image data to another information processing terminal 6 via the Internet N (step S402).

[0117] The Internet connection unit 62 of another information processing terminal 6 receives the image data via the Internet N. The Internet connection unit 62 outputs the image data to the display control unit 63. The display control unit 63 causes the display unit 64 to display the image indicated by the image data. The display unit 64 displays the image indicated by the image data (step S403). The image displayed by the display control unit 63 further includes the image of the display unit 83 of another information processing terminal 8.

[0118] Next, with reference to FIG. 8, an operation example in which another information processing terminal 6 transmits operation information will be described. The operation in FIG. 8 details the operation in step S301 of FIG. 6.

[0119] The input interface unit 61 receives the input of operation information in response to the operation of the input interface unit 61 by the user (step S501). The input interface unit 61 outputs the input operation information to the Internet connection unit 62.

[0120] The Internet connection unit 62 transmits the operation information to the smart device 7 via the Internet N (step S502).

[0121] Note that the image displayed on the display unit 64 is the image of the display unit 83 of another information processing terminal 8 photographed by the smart device 7, which is displayed by the display control unit 63. Since the image displayed by the display control unit 63 further includes the image of the display unit 83 of another information processing terminal 8, the user can operate the input interface unit 61 while checking the image of the display unit 83 of another information processing terminal 8.

[0122] Next, with reference to FIG. 9, an operation example will be described in which another information processing terminal 6 displays an image captured by the smart device 7 on the display unit 64. The operation in FIG. 9 details the operation in step S403 of FIG. 7.

[0123] The Internet connection unit 62 of another information processing terminal 6 receives image data from the smart device 7 via the Internet N (step S601). The Internet connection unit 62 outputs the image data to the display control unit 63.

[0124] The display control unit 63 receives the image data from the Internet connection unit 62. The display control unit 63 causes the display unit 64 to display the image indicated by the input image data (step S602). The display unit 64 displays the image indicated by the image data.

[0125] Next, with reference to FIG. 10, an operation example will be described in which the smart device 7 that has received operation information transmits the operation information to the information processing terminal 5. Also, the operation in FIG. 10 details the operation in step S302 of FIG. 6.

[0126] The Internet connection unit 72 receives operation information from another information processing terminal 6 via the Internet N (step S701). The Internet connection unit 72 outputs the operation information to the communication unit 71.

[0127] The communication unit 71 receives the operation information from the Internet connection unit 72. The communication unit 71 transmits the input operation information to the receiving unit 51 of the information processing terminal 5 (step S702).

[0128] Next, with reference to FIG. 11, an operation example will be described in which the smart device 7 transmits image data to another information processing terminal 6. Also, the operation in FIG. 11 details the operations from step S401 to step S402 of FIG. 7.

[0129] The imaging unit 73 captures an object to be imaged (step S801) and outputs the image data to the Internet connection unit 72. The object to be imaged further includes the display unit 83 of another information processing terminal 8.

[0130] The Internet connection unit 72 transmits the input image data to another information processing terminal 6 via the Internet N (step S802).

[0131] Next, with reference to FIG. 12, an operation example in which the information processing terminal 5 transmits operation information to another information processing terminal 8 will be described. Also, the operation in FIG. 12 details the operation in step S303 of FIG. 6.

[0132] The receiving unit 51 receives operation information indicating an operation performed on another information processing terminal 6 via the Internet N (step S901). The receiving unit 51 outputs the received operation information to the transmitting unit 52.

[0133] The transmitting unit 52 receives the operation information from the receiving unit 51. Based on the operation information, the transmitting unit 52 transmits, by wired communication, a command for connecting another information processing terminal 8 to the Internet N of another information processing terminal 6 to another information processing terminal 8 (step S902).

[0134] Next, with reference to FIG. 13, an operation example in which another information processing terminal 8 performs processing according to a command based on operation information will be described. Also, the operation in FIG. 13 details the operations from step S304 to step S305 of FIG. 6.

[0135] The communication unit 81 receives a command based on operation information from the information processing terminal 5 (step S1001). The communication unit 81 outputs the command based on the received operation information to the display control unit 82.

[0136] The display control unit 82 performs processing according to the command based on the operation information (step S1002) and causes the display unit 83 to display an image according to the processing result (step S1003).

[0137] As described above, the information processing terminal 5 controls the operation of yet another information processing terminal 8 not connected to the Internet N according to an operation made by another information processing terminal 6 connected to the Internet N. The information processing terminal 5 receives operation information indicating the operation made by the other information processing terminal 6 via the Internet N. The information processing terminal 5 transmits, based on the operation information, a command for connecting yet another information processing terminal 8 to the Internet N by the other information processing terminal 6 to yet another information processing terminal 8 by wired communication or wireless communication. In this way, by the other information processing terminal 6 transmitting the operation information to the information processing terminal 5 via the Internet N, the operation of yet another information processing terminal 8 not connected to the Internet N can be controlled. Therefore, the operation of connecting yet another information processing terminal 8 not connected to the Internet N to the Internet N can be performed by operating the information processing terminal 6. Further, by connecting the information processing terminal 5 and yet another information processing terminal 8 by wired communication or wireless communication, preparations for yet another information processing terminal 8 to receive a control operation from a remote location are completed. In this way, the preparations for yet another information processing terminal 8 to receive a control operation from a remote location can be facilitated.

[0138] [Modification Example 1 of the Second Embodiment] The display control unit of the other information processing terminal according to Modification 1 of the second embodiment corrects the image shown by the image data input from the Internet connection unit. For example, it is assumed that the shooting direction of the shooting unit 73 of the smart device 7 is not perpendicular to the screen of the display unit 83 of the other information processing terminal 8. That is, it is assumed that the shooting unit 73 of the smart device 7 is installed at a position where it can shoot the display unit 83 of the other information processing terminal 8 from an oblique direction with respect to the screen of the display unit 83. The display control unit of the other information processing terminal according to Modification 1 identifies the characteristic part of the image shown by the image data. For example, the display control unit of the other information processing terminal according to Modification 1 identifies the part corresponding to the corner of the screen of the display unit 83 of the other information processing terminal 8 as the characteristic part of the image. The display control unit of the other information processing terminal according to Modification 1 corrects the image based on the identified characteristic part of the image. For example, when the part corresponding to the corner of the screen of the display unit 83 of the other information processing terminal 8 is identified as the characteristic part of the image, the display control unit corrects the image shown by the image data so that each of the parts corresponding to the corners of the screen is arranged at the four corners of the display unit of the other information processing terminal. The display control unit of the other information processing terminal causes the corrected image to be displayed on the display unit.

[0139] Alternatively, instead of the other information processing terminal 6, the processing unit (not shown) of the smart device 7 may perform image correction. The processing unit of the smart device 7 may acquire the image data showing the screen of the display unit 83 of the other information processing terminal 8 captured by the shooting unit 73 and correct the distortion based on the characteristic part of the image. Then, the Internet connection unit 72 of the smart device 7 may transmit the image data showing the image with the distortion corrected to the other information processing terminal 6.

[0140] When an image showing the screen of the display unit 83 of still another information processing terminal 8, which has been captured by the imaging unit 73 of the smart device 7, is distorted in still another information processing terminal according to Modification 1, the display control unit causes the display unit to display an image with the distortion corrected based on the characteristic portion of the identified image. As a result, an image after the distortion has been corrected is displayed on the display unit of the other information processing terminal. Since it becomes possible to allow the user to confirm the image after the distortion has been corrected and operate the input interface unit of the other information processing terminal, the other information processing terminal according to Modification 1 can improve the convenience for the user.

[0141] [Modification 2 of the Second Embodiment] There may be a delay from when the user operates the input interface unit 61 of another information processing terminal 6 until an image corresponding to the operation of the display unit 83 of still another information processing terminal 8, which has been captured by the smart device 7, is displayed on the display unit 64 of the other information processing terminal 6. Generally, if it takes 100 milliseconds or more from when the user operates the input interface unit 61 of another information processing terminal 6 until an image corresponding to the operation is displayed on the display unit 64, the user may feel a sense of discomfort. The remote control device according to Modification 2 of the second embodiment allows the user to recognize the delay time.

[0142] The display control unit 82 of still another information processing terminal 8 causes the display unit 83 to display the system time of still another information processing terminal 8. The display control unit of the remote control device according to Modification 2 of the second embodiment causes the system time of the remote control device to be further displayed on the image indicated by the image data input from the Internet connection unit. Note that the system time is a common time between still another information processing terminal 8 and the other information processing terminal because still another information processing terminal 8 and the other information processing terminal perform time synchronization in advance.

[0143] Furthermore, since the system time of another information processing terminal is displayed on top of the image including the system time displayed on the display unit 83 of yet another information processing terminal 8, the user can compare the system time of yet another information processing terminal 8 with the system time of another information processing terminal. The other information processing terminal of this modified example enables the user to grasp the existence of a delay time by comparing the system time of yet another information processing terminal 8 with the system time of another information processing terminal. If the user cannot grasp the existence of a delay time, the screen of yet another information processing terminal 8 before the transmitted operation is reflected remains displayed on the display unit of another information processing terminal. Therefore, there is a possibility of causing an incorrect operation by the user of performing again an operation for which operation information has already been transmitted. In this modified example, by enabling the user to grasp the existence of a delay time, it is possible to reduce the possibility that the user performs an operation that has already been input.

[0144] [Modification Example 3 of the Second Embodiment] The smart device of modification example 3 of the second embodiment further includes a processing unit that performs optical character recognition. Image data is input from the imaging unit to the processing unit of the smart device. The processing unit of the smart device performs optical character recognition to recognize the characters included in the image indicated by the input image data and generates system time information indicating the system time. Alternatively, the display control unit 63 of another information processing terminal 6 may perform optical character recognition to recognize the characters included in the image indicated by the image data and generate system time information.

[0145] In this modified example, the display control unit 82 of yet another information processing terminal 8 causes the system time of yet another information processing terminal 8 to be displayed on the display unit 83. The system time is a common time between yet another information processing terminal 8 and another information processing terminal 6 by performing time synchronization in advance between yet another information processing terminal 8 and another information processing terminal 6.

[0146] Alternatively, the information processing terminal 5 may further include a display unit that displays the system time. The information processing terminal 5 may obtain system time information indicating the system time from a smart device. Further, the smart device may perform time synchronization in advance with another information processing terminal 8 and another information processing terminal 6. When the information processing terminal 5 displays the system time on the display unit, the display unit is, for example, a matrix LED in which a plurality of two or more LEDs (Light-emitting diodes) are arranged vertically and horizontally. When the information processing terminal 5 further includes a display unit that displays the system time, the smart device is arranged such that the imaging unit can image both the display unit of the information processing terminal 5 and the display unit 83 of another information processing terminal 8.

[0147] The processing unit of the smart device outputs the generated system time information to the Internet connection unit. The Internet connection unit transmits the system time information and the image data in association with each other to another information processing terminal 6.

[0148] The Internet connection unit 62 of another information processing terminal 6 that has received the system time information and the image data outputs the received system time information and the image data in association with each other to the display control unit 63. The display control unit 63 calculates the difference between the system time indicated by the input system time information and the system time of another information processing terminal 6, and causes the display unit 64 to display the image indicated by the input image data and the calculated difference.

[0149] By causing the display unit 64 to display the difference between the system time of the display unit 83 of still another information processing terminal 8 in the image of the display unit 64 and the system time of another information processing terminal 6, the other information processing terminal 6 causes the user to recognize that there is a delay time. If the user cannot recognize that there is a delay time, the screen of still another information processing terminal 8 before the transmitted operation information is reflected remains displayed on the display unit 64 of the information processing terminal 6. Therefore, there is a possibility of causing an incorrect operation by the user of performing again an operation for which the operation information has already been transmitted. On the other hand, since the remote operation system of this modification example causes the user to recognize that there is a delay time, it is possible to suppress incorrect operations by the user.

[0150] [Modification Example 4 of the Second Embodiment] Another information processing terminal of modification example 4 of the second embodiment measures the time from when operation information is input to the input interface unit of another information processing terminal until the image of the display unit 83 of still another information processing terminal 8 changes.

[0151] The input interface unit of another information processing terminal of this modification example causes the storage unit (not shown) to store the system time at which the operation information was input. Further, the display control unit of another information processing terminal calculates the number of frames until the image indicated by the image data captured after the system time at which the operation information was input changes to another image. Then, the display control unit of another information processing terminal calculates the time until the image of the display unit 83 of still another information processing terminal 8 changes based on the frame rate of the moving image captured by the smart device 7 and the calculated number of frames.

[0152] Note that the Internet connection unit 72 of the smart device 7 transmits information indicating the frame rate of the moving image captured by the smart device 7 and the image data to the remote operation device in association with each other.

[0153] Taking the case where the imaging unit 73 of the smart device 7 captures the display unit 83 of another information processing terminal 8 at 30 frames per second (fps) as an example, it will be specifically described. When a change occurs in the image captured 3 frames after the system time when the operation information was input, the display control unit of the other information processing terminal calculates 3, which is the number of frames until the image shown by the image data captured after the system time when the operation information was input changes to another image. The display control unit of the other information processing terminal calculates the time until the image of the display unit 83 of the other information processing terminal 8 changes further based on the frame rate indicated by the information indicating the frame rate and the calculated number of frames. Specifically, the display control unit of the other information processing terminal calculates 0.1 seconds (1 second ÷ 30 fps × 3 frames = 0.1 seconds). The display control unit of the other information processing terminal causes the display unit to display the image shown by the image data and the calculated delay time.

[0154] By causing the other information processing terminal to display the calculated delay time on the display unit, the user is made aware of the delay time. If the user cannot grasp that there is a delay time, the screen of the other information processing terminal 8 before the transmitted operation is reflected remains displayed on the display unit of the other information processing terminal. Therefore, there is a possibility of causing an incorrect operation by the user of performing again an operation for which the operation information has already been transmitted. On the other hand, since the remote operation system of this modification example enables the user to grasp the delay time, the other information processing terminal can suppress incorrect operations by the user.

[0155] Alternatively, the smart device 7 may calculate the delay time. The other information processing terminal transmits the system time information indicating the system time when the operation information was input to the smart device 7 in association with the operation information.

[0156] The communication unit 71 of the smart device 7 transmits the received operation information to the information processing terminal 5. Also, a processing unit (not shown) of the smart device 7 calculates the number of frames from the system time indicated by the system time information until the image indicated by the captured image data changes to another image. Then, the processing unit of the smart device 7 calculates a delay time based on the frame rate of the moving image captured by the imaging unit 73 of the smart device 7 and the calculated number of frames. The smart device 7 transmits information indicating the calculated delay time to another information processing terminal 6. Another information processing terminal 6 causes a display unit 64 to display a message indicating the delay time based on the received information indicating the delay time.

[0157] [Third Embodiment] Next, a remote operation system according to the third embodiment of the present invention will be specifically described. The third embodiment is different from the remote operation system of the second embodiment in that another information processing terminal 100 receives image control information, which is control information for displaying an image. The image based on the image control information is an image that further corresponds to the result of the display control unit 122 of another information processing terminal 120 processing a command based on the operation information from another information processing terminal 100. It is also an image displayed on the display unit 83 of another information processing terminal 120 and the display unit 64 of another information processing terminal 100. The image control information includes information about the image or information indicating the result of processing according to the command by the display control unit 122. In this embodiment, the description will focus on the differences from the second embodiment, but various examples described in the second embodiment can be applied.

[0158] Referring to FIG. 14, a configuration example of the remote operation system according to this embodiment will be described. FIG. 14 is a block diagram showing a configuration example of the remote operation system according to this embodiment. The remote operation system according to this embodiment includes an information processing terminal 9, another information processing terminal 100, a smart device 110, and still another information processing terminal 120. The information processing terminal 9 is further connected to the still another information processing terminal 120 and the smart device 110. Also, the other information processing terminal 100 is connected to the Internet N. The smart device 110 is connected to the information processing terminal 9 and the Internet N.

[0159] Referring to FIG. 14, the configuration of the other information processing terminal 100 according to this embodiment will be described in detail. The other information processing terminal 100 includes an input interface unit 61 and an Internet connection unit 102. The Internet connection unit 102 is connected to the Internet N, the input interface unit 61, and a display control unit 103. The display control unit 103 is connected to the Internet connection unit 102 and a display unit 64. Each of the input interface unit 61 and the display unit 64 of the other information processing terminal 100 in this embodiment is the same as each of the input interface unit 61 and the display unit 64 of the other information processing terminal 6 in the second embodiment shown in FIG. 4. Therefore, for each of the input interface unit 61 and the display unit 64 of the other information processing terminal 100 in this embodiment, the corresponding elements are denoted by the same reference numerals as those in FIG. 4, and the description thereof is omitted.

[0160] The Internet connection unit 102 is connected to the Internet N. The connection destination of the Internet connection unit 102 included in the Internet N includes the smart device 110. Since the processing of each configuration of the other information processing terminal 100 until starting the remote operation of the still another information processing terminal 120 and the processing of transmitting operation information are the same as those of the other information processing terminal 6 in the second embodiment, the description thereof is omitted.

[0161] The Internet connection unit 102 receives image control information, which is control information for displaying an image, from the smart device 110 via the Internet N. The Internet connection unit 102 outputs the image control information to the display control unit 103.

[0162] The display control unit 103 receives the image control information from the Internet connection unit 102. The display control unit 103 causes the display unit 64 to display an image based on the image control information.

[0163] Next, with reference to FIG. 14, the configuration of the smart device 110 of the present embodiment will be described in detail.

[0164] The smart device 110 includes a communication unit 111 and an Internet connection unit 112.

[0165] The processing when the smart device 110 receives a notification indicating that pairing processing has been performed and that it is already connected to another information processing terminal 120, and the processing when it receives a notification requesting installation are the same as those of the smart device 7 of the second embodiment. Also, the processing when the smart device 110 transmits and receives operation information is the same as that of the smart device 7 of the second embodiment. Therefore, the description of the pairing processing, the processing when the smart device 110 receives a notification indicating that it is already connected to another information processing terminal 120, the processing when it receives a notification requesting installation, and the processing when it transmits and receives operation information will be omitted.

[0166] The communication unit 111 receives the image control information. The communication unit 111 outputs the image control information to the Internet connection unit 112.

[0167] The Internet connection unit 112 receives the image control information from the communication unit 111. The Internet connection unit 112 transmits the input image control information to the other information processing terminal 100 via the Internet N.

[0168] Referring to FIG. 14, the configuration of the information processing terminal 9 of this embodiment will be described in detail.

[0169] The information processing terminal 9 includes a receiving unit 91 and a transmitting unit 92.

[0170] Since the receiving unit 91 of the information processing terminal 9 in this embodiment performs pairing processing in the same manner as the receiving unit 51 of the information processing terminal 5 in the second embodiment, the description of the pairing processing is omitted.

[0171] The receiving unit 91 receives operation information indicating an operation performed on another information processing terminal 100 via the Internet N through the smart device 110. The receiving unit 91 outputs the received operation information to the transmitting unit 92.

[0172] In addition, image control information is input to the receiving unit 91 from the transmitting unit 92. The receiving unit 91 transmits the input image control information to the smart device 110.

[0173] The transmitting unit 92 is further connected to another information processing terminal 120 when a USB terminal is inserted into the USB port of the communication unit 121 of the other information processing terminal 120. The transmitting unit 92 has two USB terminals. Each of the two USB terminals of the transmitting unit 92 is inserted into the USB port of the communication unit 121.

[0174] Operation information is input to the transmitting unit 92 from the receiving unit 91. The transmitting unit 92 transmits, via one USB terminal, a command for connecting another information processing terminal 120 to the Internet N by another information processing terminal 100 based on the input operation information to another information processing terminal 120. The transmitting unit 92 receives image control information from another information processing terminal 120 via the other USB terminal. The transmitting unit 92 outputs the image control information to the receiving unit 91.

[0175] Of the two USB terminals, at least the USB terminal that receives image control information from another information processing terminal 120 corresponds to an alternate mode that can transmit signals other than the USB standard.

[0176] Further, the transmission unit 92 does not have to include two USB terminals. Instead, the transmission unit 92 may include one USB terminal. When the transmission unit 92 includes one USB terminal, among the plurality of signal lines constituting the USB terminal, a predetermined signal line may be used to transmit a command based on operation information, and image control information may be received from the other signal lines.

[0177] With reference to FIG. 14, the configuration of still another information processing terminal 120 according to the present embodiment will be described in detail.

[0178] Still another information processing terminal 120 includes a communication unit 121. The communication unit 121 is connected to a display control unit 122. The display control unit 122 is connected to the communication unit 121 and a display unit 83. Since the display unit 83 of still another information processing terminal 120 in the present embodiment is the same as the display unit 83 of still another information processing terminal 8 in the second embodiment shown in FIG. 4, corresponding elements are denoted by the same reference numerals as those in FIG. 4, and the description thereof will be omitted.

[0179] The communication unit 121 includes two USB ports. By inserting each of the two USB terminals of the transmission unit 92 of the information processing terminal 9 into each of the two USB ports, still another information processing terminal 120 is connected to the information processing terminal 9. The communication unit 121 receives a command based on operation information from the information processing terminal 9 via one USB port. The communication unit 121 outputs the command based on operation information to the display control unit 122.

[0180] Also, image control information is input from the display control unit 122 to the communication unit 121. The communication unit 121 transmits the image control information to the information processing terminal 9 via the other USB port.

[0181] Commands based on the operation information are input from the communication unit 121 to the display control unit 122. The display control unit 122 performs processing according to the commands based on the operation information. The display control unit 122 generates image control information, which is control information for causing an image corresponding to the processing result to be displayed. Further, the image control information includes information on the image or information indicating the result of the display control unit 122 processing the commands based on the operation information. The display control unit 122 outputs the generated image control information to the display unit 83 and the communication unit 121. The image control information is further control information for causing an image displaying the result of processing commands by another information processing terminal 120 to be displayed.

[0182] The display unit 83 displays an image under the control of the display control unit 122. Specifically, the image control information is input from the display control unit 122 to the display unit 83. The display unit 83 displays an image based on the image control information. In this way, the display unit 83 displays an image indicating the result of the display control unit 122 processing the commands based on the operation information.

[0183] In this way, the information processing terminal 9 controls the operation of still another information processing terminal 120 that is not connected to the Internet N according to the operations made by another information processing terminal 100 connected to the Internet N. The information processing terminal 9 receives operation information indicating the operations made by another information processing terminal 100 via the Internet N. The information processing terminal 9 transmits, based on the operation information, a command for still another information processing terminal 120 to connect to the Internet N by the other information processing terminal 100 to still another information processing terminal 120 by wire communication or wireless communication. Therefore, by the other information processing terminal 100 transmitting the operation information to the information processing terminal 9 via the Internet N, the operation of still another information processing terminal 120 that is not connected to the Internet N can be controlled. Thereby, the operation of connecting still another information processing terminal 120 that is not connected to the Internet N to the Internet N can be performed by operating the other information processing terminal 100. Also, by connecting the information processing terminal 9 and still another information processing terminal 120 by wire communication or wireless communication, the preparation for still another information processing terminal 120 to receive a control operation from a remote location is completed. In this way, the preparation for the information processing terminal 120 to receive control from a remote location can be facilitated.

[0184] Next, with reference to FIGS. 15 to 19, an operation example of the remote operation system of the present embodiment will be described.

[0185] FIG. 15 is a sequence diagram showing an operation example of the remote operation system. FIG. 16 is a flowchart showing an operation example of another information processing terminal 100. FIG. 17 is a flowchart showing an operation example of the smart device 110. FIG. 18 is a flowchart showing an operation example of the information processing terminal 9. FIG. 19 is a flowchart showing an operation example of still another information processing terminal 120.

[0186] First, referring to FIG. 15, the operation of the other information processing terminal 100 transmitting operation information to the smart device 110, and then the other information processing terminal 120 performing processing according to the command based on the operation information, and the other information processing terminal 100 displaying an image according to the processing result will be described.

[0187] Operation information corresponding to an operation of the input interface unit 61 by the user is input to the input interface unit 61 of the other information processing terminal 100. The input interface unit 61 outputs the input operation information to the Internet connection unit 102.

[0188] The Internet connection unit 102 transmits the operation information to the smart device 110 via the Internet N (step S1101).

[0189] The Internet connection unit 112 of the smart device 110 receives the operation information from the other information processing terminal 100 via the Internet N. The communication unit 111 transmits the operation information to the receiving unit 91 of the information processing terminal 9 (step S1102).

[0190] The receiving unit 91 of the information processing terminal 9 receives the operation information indicating the operation performed on the other information processing terminal 100 via the Internet N. The transmitting unit 92 transmits, by wired communication, a command for the other information processing terminal 100 to connect to the Internet N of the other information processing terminal 120 further based on the operation information to the other information processing terminal 120 (step S1103).

[0191] Furthermore, the communication unit 121 of the other information processing terminal 120 receives the command based on the operation information from the information processing terminal 9. The display control unit 122 performs processing according to the command based on the operation information (step S1104). The display control unit 122 generates image control information, which is control information for displaying an image according to the processing result. The display control unit 122 outputs the generated image control information to the display unit 83 and the communication unit 121. The display unit 83 displays an image based on the image control information (step S1105).

[0192] The communication unit 121 receives image control information from the display control unit 122. The communication unit 121 transmits the image control information to the information processing terminal 9 (step S1106).

[0193] The transmission unit 92 further receives the image control information from another information processing terminal 120. The reception unit 91 receives the image control information from the transmission unit 92. The reception unit 91 transmits the input image control information to the smart device 110 (step S1107).

[0194] The communication unit 111 of the smart device 110 receives the image control information. The Internet connection unit 112 receives the image control information from the communication unit 111. The Internet connection unit 112 transmits the input image control information to another information processing terminal 100 via the Internet N (step S1108).

[0195] The Internet connection unit 102 of another information processing terminal 100 receives the image control information from the smart device 110 via the Internet N. The display control unit 103 receives the image control information from the Internet connection unit 102. The display control unit 103 causes the display unit 64 to display an image based on the image control information.

[0196] The display unit 64 displays an image based on the image control information (step S1109).

[0197] Next, with reference to FIG. 16, the operation of another information processing terminal 100 will be described. The operation in FIG. 16 details the operations of step S1101 and step S1109 in FIG. 15.

[0198] The input interface unit 61 receives the input of operation information corresponding to the operation of the input interface unit 61 by the user (step S1201). The input interface unit 61 outputs the input operation information to the Internet connection unit 102.

[0199] The Internet connection unit 102 transmits operation information to the smart device 110 via the Internet N (step S1202).

[0200] The Internet connection unit 102 receives image control information from the smart device 110 via the Internet N (step S1203).

[0201] The display control unit 103 receives the image control information from the Internet connection unit 102. The display control unit 103 causes the display unit 64 to display an image based on the image control information.

[0202] The display unit 64 displays an image based on the image control information (step S1204).

[0203] In addition, if another information processing terminal 100 receives new operation information before receiving the image control information, the other information processing terminal 100 may perform the following operations. The other information processing terminal 100 transmits the newly input operation information to the smart device 110. Also, regardless of the timing of receiving the operation information, each time the other information processing terminal 100 receives the image control information, it displays an image based on the image control information.

[0204] Next, with reference to FIG. 17, the operation of the smart device 110 will be described. The operation in FIG. 17 details the operations in steps S1102 and S1108 in FIG. 15.

[0205] The Internet connection unit 112 of the smart device 110 receives operation information from another information processing terminal 100 via the Internet N (step S1301). It transmits the operation information to the receiving unit 91 of the information processing terminal 9 (step S1302).

[0206] The communication unit 111 of the smart device 110 receives image control information (step S1303). The image control information is input from the communication unit 111 to the Internet connection unit 112. The Internet connection unit 112 transmits the input image control information to another information processing terminal 100 (step S1304).

[0207] Note that due to communication delays or user operations, it is assumed that after the smart device 110 transmits operation information, it receives operation information instead of image control information. Alternatively, it is assumed that after the smart device 110 transmits image control information, it receives image control information instead of operation information. Each time the smart device 110 receives operation information from another information processing terminal 100, it transmits the operation information to the information processing terminal 9. Also, each time the smart device 110 receives image control information, it transmits the image control information to another information processing terminal 100.

[0208] Next, with reference to FIG. 18, the operation of the information processing terminal 9 will be described. The operation in FIG. 18 details the operations in steps S1103 and S1107 in FIG. 15.

[0209] The receiving unit 91 of the information processing terminal 9 receives operation information indicating an operation performed by another information processing terminal 100 via the smart device 110 through the Internet N (step S1401).

[0210] The transmitting unit 92 transmits, based on the operation information, a command for another information processing terminal 100 to connect to the Internet N of yet another information processing terminal 120 to the yet another information processing terminal 120 (step S1402).

[0211] The transmitting unit 92 receives image control information from yet another information processing terminal 120 (step S1403). The image control information is input from the transmitting unit 92 to the receiving unit 91. The receiving unit 91 transmits the input image control information to the smart device 110 (step S1404).

[0212] Note that due to communication delays or user operations, it is assumed that after the information processing terminal 9 transmits a command based on operation information, it receives operation information instead of image control information. Alternatively, it is assumed that after the information processing terminal 9 transmits image control information, it receives image control information instead of operation information. Each time the information processing terminal 9 receives operation information from the smart device 110, the information processing terminal 9 further transmits a command based on the operation information to another information processing terminal 120. Also, each time the information processing terminal 9 receives image control information, the information processing terminal 9 transmits the image control information to the smart device 110.

[0213] Next, with reference to FIG. 19, the operation of another information processing terminal 120 will be further described. The operation in FIG. 19 details the operations in steps S1104, S1105, and S1106 in FIG. 15.

[0214] The communication unit 121 of another information processing terminal 120 receives a command based on operation information from the information processing terminal 9 (step S1501). The display control unit 122 performs processing according to the command based on the operation information (step S1502). The display control unit 122 generates image control information, which is control information for displaying an image according to the processing result. The display control unit 122 outputs the generated image control information to the display unit 83 and the communication unit 121. By outputting the image control information to the display unit 83, the display control unit 122 causes the display unit 83 to display an image according to the processing result (step S1503). The display unit 83 displays an image based on the image control information.

[0215] The communication unit 121 receives image control information from the display control unit 122. The communication unit 121 transmits the image control information to the information processing terminal 9 (step S1504).

[0216] Note that the display control unit 122 may stop outputting the image control information to the display unit 83. When the display control unit 122 stops outputting the image control information to the display unit 83, the display unit 83 stops displaying the image according to the processing result by the display control unit 122.

[0217] In this way, the information processing terminal 9 controls the operations of still another information processing terminal 120 that is not connected to the Internet N according to the operations made by another information processing terminal 100 connected to the Internet N. The information processing terminal 9 receives operation information indicating the operations made by another information processing terminal 100 via the Internet N. Based on the operation information, the information processing terminal 9 transmits, by wired communication or wireless communication, to still another information processing terminal 120 a command for connecting still another information processing terminal 120 to the Internet N by another information processing terminal 100. Therefore, by another information processing terminal 100 transmitting operation information to the information processing terminal 9 via the Internet N, the operations of still another information processing terminal 120 that is not connected to the Internet N can be controlled. As a result, the operation of connecting still another information processing terminal 120 that is not connected to the Internet N to the Internet N can be performed by operating another information processing terminal 100. Also, by connecting the information processing terminal 9 and still another information processing terminal 120 by wired communication or wireless communication, the preparation for still another information processing terminal 120 to receive a control operation from a remote location is completed. In this way, the preparation for still another information processing terminal 120 to receive control from a remote location can be facilitated.

[0218] The transmission unit 92 of the information processing terminal 9 according to the present embodiment receives image control information from still another information processing terminal 120. Also, the reception unit 91 transmits the image control information to another information processing terminal 100 via the smart device 110. As a result, the display control unit of another information processing terminal 100 can display an image based on the image control information. Since the display control unit of another information processing terminal 100 displays an image based on the image control information, the user can be made to operate still another information processing terminal 120 from a remote location in the same manner as when directly operating still another information processing terminal 120.

[0219] [Hardware Configuration Example] A configuration example of hardware resources for realizing each of the information processing terminals (1, 5, 9) in the above-described embodiments of the present invention using a single information processing apparatus (computer) will be described. Note that the information processing terminal may be realized using at least two information processing apparatuses physically or functionally. Further, the information processing terminal may be realized as a dedicated apparatus. Also, only some functions of the information processing terminal may be realized using an information processing apparatus.

[0220] FIG. 20 is a diagram schematically showing a hardware configuration example of an information processing apparatus capable of realizing the information processing terminal according to each embodiment of the present invention. The information processing apparatus 130 includes a communication interface 131, an input / output interface 132, an arithmetic unit 133, a storage device 134, a nonvolatile storage device 135, and a drive device 136.

[0221] For example, the receiving unit 11 and the transmitting unit 12 of the information processing terminal 1 in FIG. 1 can be realized by the communication interface 131.

[0222] The communication interface 131 is a communication means for the information processing terminal according to each embodiment to communicate with an external device by wire or / and wirelessly. Note that when the information processing terminal is realized using at least two information processing apparatuses, they may be connected so as to be able to communicate with each other via the communication interface 131.

[0223] The input / output interface 132 is a man-machine interface such as a keyboard which is an example of an input device and a display as an output device.

[0224] The arithmetic unit 133 is realized by a general-purpose CPU (Central Processing Unit), a microprocessor, or the like, which is an arithmetic processing device, or a plurality of electric circuits. The arithmetic unit 133 can, for example, read various programs stored in the nonvolatile storage device 135 into the storage device 134 and execute processing according to the read programs.

[0225] The storage device 134 is a memory device such as a RAM (Random Access Memory) that can be referenced by the arithmetic unit 133, and stores programs and various data. The storage device 134 may be a volatile memory device.

[0226] The non-volatile storage device 135 is a non-volatile storage device such as a ROM (Read Only Memory) or a flash memory, and can store various programs and data.

[0227] The drive device 136 is a device that processes reading and writing of data to a recording medium 137 described later, for example.

[0228] The recording medium 137 is an arbitrary recording medium capable of recording data, such as an optical disk, a magneto-optical disk, or a semiconductor flash memory, for example.

[0229] Each embodiment of the present invention may configure an information processing terminal by an information processing device 130 illustrated in FIG. 20, for example. And each embodiment of the present invention may be realized by supplying a program capable of realizing the functions described in the above embodiments to this information processing terminal.

[0230] In this case, the embodiment can be realized by the arithmetic unit 133 executing the program supplied to the information processing terminal. Also, it is possible to configure not all of the information processing terminal but some of the functions by the information processing device 130.

[0231] Furthermore, the above program may be recorded on a recording medium 137, and the information processing terminal may be configured such that the above program is appropriately stored in the non-volatile memory device 135 at the shipping stage or the operation stage of the information processing terminal. In this case, as the method for supplying the above program, a method of installing it inside the information processing terminal using an appropriate jig may be adopted at the manufacturing stage before shipping or the operation stage. Also, as the method for supplying the above program, a general procedure such as a method of downloading from the outside via a communication line such as the Internet may be adopted.

[0232] Note that each of the above-described embodiments is a preferred embodiment of the present invention, and various modifications can be made without departing from the gist of the present invention.

Explanation of Reference Numerals

[0233] 1, 5, 9 Information processing terminal 11, 51, 91 Receiving unit 12, 52, 92 Transmitting unit 2, 6, 100 Information processing terminal 21, 61 Input interface unit 22, 62, 102 Internet connection unit 63, 103 Display control unit 64 Display unit 3, 7, 110 Smart device 31, 71, 111 Communication unit 32, 72, 112 Internet connection unit 73 Photographing unit 4, 8, 120 Information processing terminal 41, 81, 121 Communication unit 82, 122 Display control unit 83 Display unit N Internet 130 Information processing device 131 Communication interface 132 Input / output interface 133 Arithmetic unit 134 Storage device 135 Non-volatile memory device 136 Drive device 137 Recording medium

Claims

1. A remote operation method for an information processing terminal that controls the operation of yet another information processing terminal not connected to the Internet according to an operation made to another information processing terminal connected to the Internet, receiving operation information indicating the operation made to the other information processing terminal via the Internet, transmitting, based on the operation information, a command corresponding to an operation for starting the connection of the yet another information processing terminal to the Internet by the other information processing terminal to the yet another information processing terminal by wired communication or wireless communication, image data showing a screen of the display unit of the yet another information processing terminal, photographed by a device installed so as to be able to photograph the display unit of the yet another information processing terminal, the image data showing a screen in which a system time that is a common time between the other information processing terminal and the yet another information processing terminal is displayed, transmitting the image data to the other information processing terminal, the other information processing terminal displaying both the image shown by the image data and the system time of the other information processing terminal on the other information processing terminal A remote operation method.

2. In the image data showing the screen of the display unit of the yet another information processing terminal, correcting the distortion of the image shown by the image data, transmitting, to the other information processing terminal, image data showing an image with corrected distortion The remote operation method according to claim 1, characterized in that.

3. Notifying the other information processing terminal that the yet another information processing terminal has connected to the Internet The remote operation method according to claim 1 or claim 2, characterized in that.

4. Notifying the other information processing terminal that the preparation for remote operation of the yet another information processing terminal has been completed The remote operation method according to any one of claims 1 to 3, characterized in that.

5. Image control information, which is control information for displaying an image showing the result of processing the command by the further other information processing terminal, is received from the further other information processing terminal, and the image control information is transmitted to the other information processing terminal. The remote operation method according to claim 1, characterized in that.

6. A remote operation program for an information processing terminal that controls the operation of a further other information processing terminal not connected to the Internet according to an operation performed on another information processing terminal connected to the Internet, causing a computer to a receiving function for receiving, via the Internet, operation information indicating the operation performed on the other information processing terminal; a transmission function for transmitting, by wired communication or wireless communication, to the further other information processing terminal, a command corresponding to an operation for starting the connection of the further other information processing terminal to the Internet based on the operation information; image data showing a screen of the display unit of the further other information processing terminal, captured by a device installed so as to be able to capture the display unit of the further other information processing terminal, the image data showing a screen in which a system time, which is a common time between the other information processing terminal and the further other information processing terminal, is displayed, is transmitted to the other information processing terminal; a display function for causing the other information processing terminal to display both the image indicated by the image data and the system time of the other information processing terminal on the other information processing terminal; A remote operation program for realizing.

7. An information processing terminal that controls the operation of a further other information processing terminal not connected to the Internet according to an operation performed on another information processing terminal connected to the Internet, a receiving unit that receives, via the Internet, operation information indicating the operation performed on the other information processing terminal; A transmission unit that transmits, by wired communication or wireless communication, to the still another information processing terminal, a command corresponding to an operation for starting the connection of the still another information processing terminal to the Internet by the other information processing terminal, based on the operation information; comprising; The transmission unit transmits, to the other information processing terminal, image data showing a screen of a display unit of the still another information processing terminal, which is captured by a device installed so as to be able to capture the display unit of the still another information processing terminal, and which shows a screen in which a system time that is a common time between the other information processing terminal and the still another information processing terminal is displayed; An information processing terminal in which an image shown by the image data and the system time of the other information processing terminal are both displayed by the other information processing terminal.

Images