Remote operation assistance system and method

Abstract

According to one aspect of this invention, when executed by a system that connects a user terminal and an operator terminal over a network and transfers an operation of the operator to a user in a state in which a display screen is shared, first, operation information in which the movement of a finger of the operator is reflected is detected in the operator terminal, and the detected operation information is transmitted from the operator terminal to the user terminal over the network. With regard to this, the user terminal receives the operation information transmitted from the operator terminal, generates a stimulus control signal for generating the same movement as the finger of the operator in the finger of the user on the basis of the received operation information, and presents a stimulus to a body part related to the movement of the finger of the user in response to the generated stimulus control signal.

Description

Remote operation support system and method

[0001] One aspect of the present invention relates to a remote operation support system and a remote operation support method used to remotely support the operation of a smartphone, for example.

[0002] For example, one service that supports users who are unfamiliar with smartphone operation is known, in which the user's smartphone is connected to an operator terminal at a service center via a network, and the operator provides operation instructions while the screen is shared between the two terminals (see, for example, Non-Patent Document 1). By using this service, the user can receive operation assistance from the operator while at home, as if they were face-to-face.

[0003] "Anshin Remote Support," NTT Docomo, Internet <URL: https: / / www.docomo.ne.jp / service / remote_support / >

[0004] However, in this type of conventional service, the user confirms the operation method by visually watching the screen operated by the operator. Therefore, when the user later tries to perform the operation that the operator saw with their own eyes, the user's finger movements may differ from the operator's operation, making it difficult to reproduce the operation. In particular, for users who have difficulty moving their fingers freely due to disabilities or other factors, it is difficult to reproduce the operation of the operator that they saw with their own finger movements.

[0005] This invention has been made in light of the above circumstances, and aims to provide a technology that enables even users who are unfamiliar with operation or who have difficulty moving their fingers at will to reliably operate an information terminal using their fingers.

[0006] In order to solve the above problems, one aspect of the remote operation support system or remote operation support method according to the present invention is a system that connects a user terminal and an operator terminal via a network and transmits an operator's operation to a user while sharing a display screen, and when executed, the operator terminal first detects motion information reflecting the movement of the operator's fingers, and transmits the detected motion information from the operator terminal to the user terminal via the network. In response to this, the user terminal receives the motion information transmitted from the operator terminal, generates a stimulus control signal based on the received motion information to cause the user's fingers to move in the same way as the movement of the operator's fingers, and presents a stimulus to a body part related to the movement of the user's fingers in accordance with the generated stimulus control signal.

[0007] According to one aspect of the present invention, motion information reflecting the movement of an operator's fingers, such as an electromyographic signal, is detected and transmitted to a user terminal, and the user terminal applies a stimulus, such as an electrical stimulus, to a part of the user's finger associated with the movement of the user's finger based on the motion signal, so as to cause the user's finger to move in the same way as the operator's finger. As a result, the movement of the operator's finger associated with the operation is transmitted directly to the user's finger, allowing the user to directly recognize the movement of the operator's finger at a remote location as the movement of his or her own finger.

[0008] In other words, according to one aspect of the present invention, it is possible to provide technology that enables even users who are unfamiliar with operation or who have difficulty moving their fingers freely at will to reliably operate an information terminal using their fingers.

[0009] Fig. 1 is a diagram showing an example of the configuration of a remote operation support system according to an embodiment of the present invention. Fig. 2 is a block diagram showing an example of the configuration of an operator terminal in the remote operation support system shown in Fig. 1. Fig. 3 is a block diagram showing an example of the configuration of a user terminal in the remote operation support system shown in Fig. 1. Fig. 4 is a flowchart showing an example of the processing procedure and processing content of the operator terminal and user terminal shown in Figs. 2 and 3. Fig. 5A is a diagram showing an example of an operation to be supported. Fig. 5B is a diagram showing another example of an operation to be supported.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [One Embodiment] (Configuration Example) (1) System FIG. 1 is a diagram showing an example of the configuration of a remote operation support system according to one embodiment of the present invention.

[0012] A remote operation support system according to one embodiment of the present invention connects a user terminal UT used by a user receiving support with an operator terminal OT used by an operator providing support via a network NW, and in this state a display screen is shared between the user terminal UT and the operator terminal OT, and the operator supports the user in operating the terminal using their fingers while looking at this display screen.

[0013] An operator and a user wear gloves HE1 and HE2 for operation, respectively. A sensor unit and a stimulus presentation unit are installed or connected to the gloves HE1 and HE2, respectively.

[0014] The sensor unit is composed of electromyographic sensors ES1 and ES2 and their detection electrodes. The detection electrodes include, for example, multiple sets of two electrodes that form a pair, and these sets are placed in the glove at locations corresponding to each finger. The electromyographic sensors ES1 and ES2 measure the surface electromyography (EMG) of the skin, and measure the skin surface potential difference that appears between the two electrodes for each set of electrodes as an electromyographic signal.

[0015] More specifically, the EMG sensors ES1 and ES2 detect amplitude changes corresponding to muscle activity associated with the finger movement accompanying the swipe operation. The EMG sensors ES1 and ES2 then transmit the detected EMG signals together with identification information of the electrode set that detected them to the operator terminal OT and the user terminal UT via, for example, a signal cable or a wireless line that employs a low-power wireless data transmission standard such as Bluetooth (registered trademark).

[0016] The stimulation presentation unit is composed of electrical stimulation generators ED1 and ED2 and stimulation presentation electrodes. The stimulation presentation electrodes include, for example, multiple sets of two electrodes that make a pair. Each electrode set is placed in a position inside the glove corresponding to the muscle that moves each finger.

[0017] The electrical stimulation generators ED1 and ED2 individually apply electrical stimulation to the muscles that move each finger, generating a low-frequency stimulation signal suitable for simulated muscle contraction by superimposing a stimulation frequency of 15 to 200 Hz on a reference frequency of, for example, several to 20 kHz. The reference frequency, stimulation frequency, and signal strength of the electrical stimulation signal can be arbitrarily controlled by electrical stimulation control signals output from the operator terminal OT and the user terminal UT, and can be arbitrarily set depending on the type and condition of the muscle to be stimulated.

[0018] The network NW comprises, for example, a wide area network with the Internet at its core, and an access network for accessing this wide area network. Examples of the access network include a public communication network using wired or wireless connections, a local area network (LAN) using wired or wireless connections, and a cable television (CATV) network. When the business support system is operated within a company or office, for example, the network NW is configured as an in-house network such as a LAN or wireless LAN.

[0019] (2) Operator Terminal OT FIG. 2 is a block diagram showing an example of the hardware and software configuration of the operator terminal OT.

[0020] The operator terminal OT is, for example, a smartphone, and includes a control unit 11 that uses a hardware processor such as a central processing unit (CPU). A storage unit having a program storage unit 12 and a data storage unit 13, an input / output interface (hereinafter, the interface will be abbreviated as I / F) unit 14, and a communication I / F unit 5 are connected to the control unit 11 via a bus (not shown).

[0021] As the operator terminal OT, a PDA terminal equipped with a touch panel or a notebook personal computer may be used in addition to a smartphone.

[0022] A touch panel type input / output device is connected to the input / output I / F unit 14. The touch panel type input / output device is configured by arranging an input device 162, which is, for example, a piezoelectric or electrostatic input sheet, on the display surface of a display device 161 that uses, for example, a liquid crystal or organic EL.

[0023] The aforementioned electromyographic sensor ES1 and electrical stimulation generator ED1 are also connected to the input / output I / F unit 14. Note that E11 and E12 are detection electrodes and stimulation presentation electrodes connected to the electromyographic sensor ES1 and electrical stimulation generator ED1, respectively.

[0024] The communication I / F unit 15 transmits and receives information data to and from the user terminal UT via the network NW in accordance with a communication protocol defined by the network NW.

[0025] The program storage unit 12 is configured by combining, for example, a nonvolatile memory such as a solid-state drive (SSD) as a storage medium that can be written to and read from at any time, and a nonvolatile memory such as a read-only memory (ROM), and stores middleware such as an operating system (OS), as well as application programs required to execute various control processes according to an embodiment. Hereinafter, the OS and each application program will be collectively referred to as the program.

[0026] The data storage unit 13 is, for example, a storage medium that combines a non-volatile memory such as an SSD that can be written to and read from at any time with a volatile memory such as a RAM (Random Access Memory), and is used to temporarily store transmitted and received data and data generated in the process of providing operational assistance to the user using electrical stimulation.

[0027] The control unit 11 includes, as processing functions necessary for implementing one embodiment, a screen sharing control processing unit 111, a myoelectric signal reception processing unit 112, an electrical stimulation control processing unit 113, and a myoelectric signal transmission processing unit 114. These processing units 111 to 114 are all realized by causing a hardware processor of the control unit 11 to execute application programs stored in the program storage unit 12.

[0028] The screen sharing control processing unit 111 shares display screen data used for operational support with the user terminal UT when a communication link is established between the user terminal UT and the screen sharing control processing unit 111 .

[0029] Myoelectric signal reception processing unit 112 receives the user's myoelectric signals transmitted from user terminal UT via communication I / F unit 15 and passes the received user's myoelectric signals to electrical stimulation control processing unit 113 .

[0030] The electrical stimulation control processing unit 113 generates an electrical stimulation control signal in response to the user's electromyographic signal to cause the operator's finger to move in the same way as the user's finger as represented by the electromyographic signal, and outputs the generated electrical stimulation control signal from the input / output I / F unit 14 to the electrical stimulation generator ED1.

[0031] The myoelectric signal transmission processing unit 114 receives the operator's myoelectric signal detected by the myoelectric sensor ES1 from the input / output I / F unit 14, and transmits the received operator's myoelectric signal from the communication I / F unit 15 to the user terminal UT.

[0032] (3) User Terminal UT FIG. 3 is a block diagram showing an example of the hardware and software configuration of the user terminal UT.

[0033] The user terminal UT is a smartphone, similar to the operator terminal OT described above. Note that the user terminal UT may also be an information terminal using a touch panel, such as a PDA.

[0034] The user terminal UT comprises a control unit 21 using a hardware processor such as a central processing unit (CPU), a storage unit having a program storage unit 22 and a data storage unit 23, an input / output I / F unit 24, and a communication I / F unit 25, all connected via a bus.

[0035] The input / output I / F unit 24 is connected to a touch panel type input / output device in which a sheet-like input device 162 is arranged on the display surface of a display device 161 .

[0036] An electromyographic sensor ES2 and an electrical stimulation generator ED2 are also connected to the input / output I / F unit 14. A detection electrode E21 is connected to the electromyographic sensor ES2, and a stimulation presentation electrode E22 is connected to the electrical stimulation generator ED2.

[0037] The communication I / F unit 25 transmits and receives information data to and from the operator terminal OT via the network NW in accordance with a communication protocol defined by the network NW.

[0038] The program storage unit 22 is configured by combining, for example, a nonvolatile memory such as an SSD as a storage medium that can be written to and read from at any time, and a nonvolatile memory such as a ROM, and stores middleware such as an OS as well as application programs required to execute various control processes according to an embodiment. Hereinafter, the OS and each application program will be collectively referred to as a program.

[0039] The data storage unit 23 is, for example, a storage medium that combines a non-volatile memory such as an SSD that can be written to and read from at any time with a volatile memory such as a RAM, and is used to temporarily store transmitted and received data and data generated in the process of receiving operational assistance using electrical stimulation.

[0040] The control unit 21 includes, as processing functions necessary to implement one embodiment, a screen sharing control processing unit 211, a myoelectric signal transmission processing unit 212, a myoelectric signal reception processing unit 213, and an electrical stimulation control processing unit 214. These processing units 211 to 214 are all realized by causing a hardware processor of the control unit 21 to execute application programs stored in the program storage unit 22.

[0041] The screen sharing control processing unit 211 shares display screen data used to receive operational support with the operator terminal OT when a communication link is established with the operator terminal OT.

[0042] The myoelectric signal transmission processing unit 212 receives the user's myoelectric signal detected by the myoelectric sensor ES2 from the input / output I / F unit 24, and transmits the received user's myoelectric signal from the communication I / F unit 25 to the operator terminal OT.

[0043] The myoelectric signal receiving processor 213 receives the operator's myoelectric signal transmitted from the operator terminal OT via the communication I / F unit 25 and passes the received operator's myoelectric signal to the electrical stimulation control processor 214 .

[0044] The electrical stimulation control processing unit 214 generates an electrical stimulation control signal in response to the operator's electromyographic signal to cause the user's finger to move in the same way as the operator's finger as represented by the electromyographic signal, and outputs the generated electrical stimulation control signal from the input / output I / F unit 24 to the electrical stimulation generator ED2.

[0045] (Example of Operation) Next, an example of operation of the remote operation support system configured as above will be described.

[0046] FIG. 4 is a flowchart showing an example of the processing procedure and processing contents of operation support control executed by the respective control units 11, 21 of the operator terminal OT and the user terminal UT of the remote operation support system.

[0047] (1) Advance preparation for operation assistance When receiving operation assistance from the user terminal UT, the user first puts on the glove HE2 on the hand that will perform the operation. Similarly, the operator puts on the glove HE1 on the hand that will perform the operation. As shown in Figure 1, the gloves HE1 and HE2 are integrally provided with electromyographic sensors ES1 and ES2, and are connected to electrical stimulation generators ED1 and ED2 via signal cables.

[0048] The myoelectric sensors ES1, ES2 and the electrical stimulation generators ED1, ED2 may both be integrally installed in the gloves HE1, HE2, or may be connected by a signal cable.

[0049] Furthermore, the electromyographic sensors ES1, ES2 and the electrical stimulation generators ED1, ED2 are paired in advance with the operator terminal OT and the user terminal UT so that communication is possible via Bluetooth, for example.

[0050] (2) Launching an Application When a user wishes to receive guidance from an operator on how to operate a smartphone, the user inputs a request for operational assistance by, for example, tapping an icon on the user terminal UT. When the control unit 11 of the user terminal UT detects the request for operational assistance in step S10, it launches an application, executes processing to establish a communication link with the operator terminal OT, and then transmits the request for operational assistance to the operator terminal OT.

[0051] In response to this, the control unit 21 of the operator terminal OT establishes a communication link with the user terminal UT in response to access from the user terminal UT, and then, upon receiving the above-mentioned operation assistance request in step S11, launches an application and notifies the operator of this.

[0052] (3) Trial Operation by User In response to the trial operation by the user, in step S12, the control unit 21 of the user terminal UT reads out display screen data to be operated from the data storage unit 13 or downloads it from a website, and displays it on the display device 261. At the same time, under the control of the screen sharing control processing unit 211, the display screen data is transmitted from the communication I / F unit 25 to the operator terminal OT.

[0053] In response to this, in step S13, the control unit 11 of the operator terminal OT, under the control of the screen sharing control processing unit 111, receives the display screen data transmitted from the user terminal UT via the communication I / F unit 15. Then, the received display screen data is displayed on the display device 161.

[0054] The display screen data may be, for example, map data, but may also be image data such as photographs or text data.

[0055] Now, suppose that in this state, the user performs a swipe operation on the display screen data by moving the fingers wearing the glove HE2 on the input device 262 of the user terminal UT. Figures 5A and 5B show, as examples of such operations, a pinch-in operation and a pinch-out operation using two fingers simultaneously. Other examples of such operations include an operation using three fingers simultaneously, a drag operation using one finger, a flick operation, a double-tap operation, and the like.

[0056] When the control unit 21 of the user terminal UT detects the user's trial operation in step S14, under the control of the myoelectric signal transmission processing unit 212, the control unit 21 acquires, via the input / output I / F unit 24 in step S15, a myoelectric signal (including identification information of the electrode set that detected the signal) corresponding to the finger movement associated with the trial operation, detected by the myoelectric sensor ES2. Then, the myoelectric signal transmission processing unit 212 transmits the acquired myoelectric signal of the user from the communication I / F unit 25 to the operator terminal OT in step S16. At the same time, under the control of the screen sharing control processing unit 211, the control unit 21 of the user terminal UT transmits the operated display screen data from the communication I / F unit 25 to the operator terminal OT.

[0057] In response to this, the control unit 11 of the operator terminal OT, under the control of the screen sharing control processing unit 111, receives the display screen data transmitted from the user terminal UT via the communication I / F unit 15 in step S17, and then in step S18, updates the display screen data displayed on the display device 161 to the received display screen data.

[0058] At the same time, the control unit 11 of the operator terminal OT, under the control of the myoelectric signal receiving processing unit 112, receives the user's myoelectric signal transmitted from the user terminal UT via the communication I / F unit 15 in step S17.

[0059] Upon receiving the user's myoelectric signal, the control unit 11 of the operator terminal OT, under the control of the electrical stimulation control processing unit 113, generates an electrical stimulation control signal in step S19 to induce the same movement in the operator's finger as the movement of the user's finger represented by the user's myoelectric signal. The electrical stimulation control signal includes values ​​specifying the intensity (amplitude) and application time of the electrical stimulation, and an identification signal specifying the electrode set corresponding to the finger to which the electrical stimulation is to be applied. The electrical stimulation control processing unit 113 then outputs the generated electrical stimulation control signal from the input / output I / F unit 14 to the electrical stimulation generator ED1.

[0060] As a result, the electrical stimulation generator ED1 generates an electrical stimulation signal of the strength specified by the electrical stimulation control signal for the specified duration, and the generated electrical stimulation signal is applied to the set of stimulation presentation electrodes corresponding to the finger to be moved. Therefore, electrical stimulation is applied to the muscles that move the operator's finger, which is the same as the finger used by the user in the trial operation, thereby moving the operator's finger. In other words, the user's finger movement is transferred to the operator's finger movement. Therefore, the operator can recognize the finger movement associated with the user's trial operation from his or her own finger movement, and can directly confirm any points in the user's trial operation that need to be corrected.

[0061] (4) Support for User Operation The operator refers to the confirmation result of the user's finger movement and performs a model operation, such as a pinch-in operation or a pinch-out operation, on the display screen of the operator terminal OT. When the control unit 11 of the operator terminal OT detects the model operation in step S20, under the control of the myoelectric signal transmission processing unit 114, acquires, via the input / output I / F unit 24 in step S21, a myoelectric signal (including identification information of the electrode set that detected the signal) corresponding to the finger movement associated with the model operation, which is detected by the myoelectric sensor ES1. Then, the myoelectric signal transmission processing unit 114 transmits the acquired user's myoelectric signal from the communication I / F unit 15 to the user terminal UT in step S22.

[0062] In response to this, the control unit 21 of the user terminal UT, under the control of the myoelectric signal receiving processing unit 213, receives the operator's myoelectric signal transmitted from the operator terminal OT via the communication I / F unit 25 in step S23.

[0063] Upon receiving the operator's myoelectric signal, the control unit 21 of the user terminal UT, under the control of the electrical stimulation control processing unit 214, generates an electrical stimulation control signal in step S24 for causing the user's finger to move in the same way as the operator's finger represented by the operator's myoelectric signal. This electrical stimulation control signal includes values ​​specifying the intensity (amplitude) and application time of the electrical stimulation, and an identification signal specifying the electrode set corresponding to the finger to which the electrical stimulation is to be applied. The electrical stimulation control processing unit 214 then outputs the generated electrical stimulation control signal from the input / output I / F unit 24 to the electrical stimulation generator ED2.

[0064] As a result, the electrical stimulation generator ED2 generates an electrical stimulation signal of the strength specified by the electrical stimulation control signal for the specified duration, and the generated electrical stimulation signal is applied to the set of stimulation presentation electrodes corresponding to the finger to be moved. Therefore, electrical stimulation is applied to the muscles that move the user's finger, which is the same as the finger used by the operator for the operation, thereby moving the user's finger. In other words, the operator's model finger movement is transferred to the user's finger. Therefore, the user can directly recognize the finger movement associated with the operator's model operation as their own finger movement.

[0065] When the display screen is operated in accordance with the movement of the user's finger, the display screen data after this operation is transmitted from the communication I / F unit 25 to the operator terminal OT in step S25 under the control of the screen sharing control processing unit 211.

[0066] In response to this, the control unit 11 of the operator terminal OT, under the control of the screen sharing control processing unit 111, receives the display image data transmitted from the user terminal UT via the communication I / F unit 15 in step S26, and in step S27, outputs the received display image data from the input / output I / F unit 14 to the display device 161 for display.

[0067] Therefore, the operator can check the results of the user's operation by looking at the display screen data.

[0068] (5) Determination of Assistance Control In step S28, the control unit 21 of the user terminal UT monitors whether the user performs an operation to terminate the operational assistance. If the user does not perform an operation to terminate the operational assistance, the control unit 21 returns to step S14 and repeatedly executes the series of processes for operational assistance in steps S14 to S28. On the other hand, if a request to terminate the operational assistance is detected, the control unit 21 transmits the request to terminate the operational assistance from the communication I / F unit 25 to the operator terminal OT, and then closes the application to terminate the operational assistance process.

[0069] In response to this, the control unit 11 of the operator terminal OT monitors in step S29 whether or not the operation assistance end request has been received from the user terminal UT. If the operation assistance end request has not been received, the process returns to step S17, and operation assistance control continues through steps S17 to S29. On the other hand, if the operation assistance end request has been received, the control unit 11 of the operator terminal OPT notifies the operator of this fact with a message, closes the application, and ends the operation assistance.

[0070] (Actions and Effects) As described above, in one embodiment, for example, the finger movement accompanying a swipe operation that is a model for the operator is detected by the electromyographic sensor ES1, and the detected electromyographic signal is transmitted from the operator terminal OT to the user terminal UT. In response to this, upon receiving the electromyographic signal, the user terminal UT generates an electrical stimulation control signal for causing the user's finger to move in the same way as the model finger movement represented by the electromyographic signal, and supplies this to the electrical stimulation generator ED2. The electrical stimulation generator ED2 then applies electrical stimulation to the user's finger, causing the finger to move.

[0071] Therefore, for users who are unfamiliar with smartphone operation or who have difficulty moving their fingers freely due to disabilities, the operator can directly demonstrate the operations as finger movements, enabling users to reliably perform relatively difficult operations such as swiping.

[0072] In one embodiment, prior to providing operational assistance to the user, the user first performs an operation and an electromyographic signal representing the finger movement at that time is transmitted from the user terminal UT to the operator terminal OT. The operator terminal OT then generates an electrical stimulation control signal based on the electromyographic signal and applies an electrical stimulation to the operator's finger, thereby directly communicating the user's finger movement to the operator.

[0073] As a result, the operator can understand the corrections needed for the user's finger movements and then provide a model for the user. This makes it possible to provide more accurate and efficient support for smartphone operation based on the characteristics of each user's finger movements.

[0074] [Other Embodiments] (1) In one embodiment, myoelectric signals are detected as motion information reflecting the finger movements associated with the operator's model operation. However, instead of this, for example, the position of the fingertip and its acceleration in multiple axial directions may be detected, and the user's finger movements may be controlled based on the detected fingertip positions and accelerations. In this case, an actuator or the like may be used to control the user's finger movements.

[0075] (2) In one embodiment, electromyography signals are detected as motion information reflecting the finger movements associated with the user's operation. However, in addition to this, for example, electrodermal activity may be detected. In this case, the degree of difficulty of the user in the operation is estimated based on the detected electrodermal activity, and the result is presented to the operator. This enables the operator to provide more detailed support by adjusting the speed of the example operation, for example.

[0076] (3) In addition, various modifications can be made to the types and functional configurations of the operator terminal and user terminal, their processing procedures and processing contents, the types of operations to be supported, etc., without departing from the spirit of this invention.

[0077] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0078] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0079] UT...user terminal OT...operator terminal NW...network ES1, ES2...electromyogram sensors E11, E21...detection electrodes ED1, ED2...electrical stimulation generators E12, E22...stimulation presentation electrodes 11, 21...controller 12, 22...program storage unit 13, 23...data storage unit 14, 24...input / output I / F unit 15, 25...communication I / F unit 161, 261...display device 162, 262...input device 111, 211...screen sharing control processor 112, 213...electromyogram signal reception processor 113, 214...electrical stimulation control processor 114, 212...electromyogram signal transmission processor

Claims

1. A remote operation support system that connects a user terminal and an operator terminal via a network and transmits operator operations to a user while sharing a display screen, comprising: a first sensor unit that detects operator movement information reflecting movements of the operator's fingers; a transmission processing unit that is provided in the operator terminal and transmits the detected operator movement information to the user terminal via the network; a reception processing unit that is provided in the user terminal and receives the operator movement information transmitted from the operator terminal; a stimulus control unit that is provided in the user terminal and generates a user stimulus control signal based on the received operator movement information, for causing the user's fingers to move in the same way as the movement of the operator's fingers; and a first stimulus presentation unit that presents a stimulus to a body part related to the movement of the user's fingers in response to the generated user stimulus control signal.

2. A remote operation support system as described in claim 1, further comprising: a second sensor unit that detects user motion information reflecting the movement of the user's fingers; a second transmission processing unit that is provided in the user terminal and that transmits the detected user motion information to the operator terminal via the network; a second reception processing unit that is provided in the operator terminal and that receives the user motion information transmitted from the user terminal; a second stimulus control unit that is provided in the operator terminal and that generates an operator stimulus control signal based on the received user motion information to cause the operator's fingers to move in the same way as the movement of the user's fingers; and a second stimulus presentation unit that presents a stimulus to a body part related to the movement of the operator's fingers in response to the generated operator stimulus control signal.

3. The remote operation support system of claim 1, wherein the first sensor unit detects electromyographic signals reflecting the movement of the operator's fingers as the operator movement information, and the first stimulation presentation unit applies electrical stimulation to the muscle areas that move the user's fingers.

4. The remote operation support system of claim 2, wherein the second sensor unit detects myoelectric signals reflecting the movement of the user's fingers as the user motion information, and the second stimulation presentation unit applies electrical stimulation to the muscle areas that move the operator's fingers.

5. A remote operation support system as described in claim 1, wherein the first sensor unit detects the operator operation information reflecting the movement of the operator's fingers related to a swipe operation.

6. The remote operation support system according to claim 2, wherein the second sensor unit detects the user action information reflecting the movement of the user's fingers associated with a swipe operation.

7. A remote operation support method executed by a system that connects a user terminal and an operator terminal via a network and transmits operator operations to a user while sharing a display screen, the remote operation support method comprising the steps of: detecting motion information reflecting the movement of the operator's fingers; transmitting the detected motion information from the operator terminal to the user terminal via the network; receiving the motion information transmitted from the operator terminal at the user terminal; generating a stimulation control signal based on the received motion information, for causing the user's fingers to move in the same manner as the movement of the operator's fingers; and presenting a stimulation to a body part related to the movement of the user's fingers in response to the generated stimulation control signal.

Images