Control system, control method, and computer program

Abstract

This control system comprises: an image display device that displays a moving image indicating a prescribed action; a myoelectric stimulation generation device that generates a myoelectric stimulation for stimulating the muscles of a subject; and a myoelectric stimulation control device that controls the myoelectric stimulation generation device so as to generate a myoelectric stimulation corresponding to the prescribed action. The myoelectric stimulation control device causes a delay such that the timing of generation of the myoelectric stimulation by the myoelectric stimulation generation device is later than the timing of display of the moving image on the image display device.

Description

Control System, Control Method, and Computer Program 【0001】 The present invention relates to a control system, a control method, and a computer program. 【0002】 Conventionally, a technique has been proposed to reproduce the movements of a source person in a destination person by using Electrical Muscle Stimulation (EMS) technology. For example, Non-Patent Document 1 discloses that by providing a destination person with video and electromyostimulation, the movements of the source person can be reproduced in the destination person with higher accuracy compared to the case of providing only video. 【0003】 "Motion Ability Transfer Technology for Promoting Muscle Coordination Movements (One Form of Motion Ability Transfer Technology)", [online], Technical Explanation Supporting IOWN, [searched on November 26, 2024], Internet <URL: https: / / www.rd.ntt / iown_tech / post_39.html> 【0004】 However, even when a destination person is provided with video and electromyostimulation, there is variation in the degree of reproduction obtained by different people. The present invention has been made in view of the above circumstances, and provides a technology that enables higher accuracy in reproducing movements by providing a destination person with video and electromyostimulation. 【0005】 One aspect of the present invention includes an image display device that displays a moving image indicating a predetermined action, an electromyostimulation generation device that generates electromyostimulation for stimulating the muscles of a subject, and an electromyostimulation control device that controls the electromyostimulation generation device to generate electromyostimulation corresponding to the predetermined action. The electromyostimulation control device causes a delay so that the timing of generating the electromyostimulation in the electromyostimulation generation device is delayed compared to the timing of displaying the moving image in the image display device, and is a control system. 【0006】One aspect of the present invention is the control system described above, further comprising: a sensor that acquires motion-related information which is information related to the movements of the subject; and a delay information acquisition device that acquires information indicating the delay based on the motion-related information, wherein the electromyography stimulation control device generates the delay acquired by the delay information acquisition device. 【0007】 One aspect of the present invention is a control method comprising: an image display step of displaying a moving image showing a predetermined action; an electromyographic stimulation generation step of generating electromyographic stimulation that stimulates the muscles of a subject; and an electromyographic stimulation control step of controlling the execution timing of the electromyographic stimulation generation step to generate electromyographic stimulation corresponding to the predetermined action, wherein the electromyographic stimulation control step causes a delay such that the timing of generating the electromyographic stimulation is delayed compared to the timing of displaying the moving image. 【0008】 One aspect of the present invention is an electromyography (EMG) control device in an information system comprising: an image display device that displays a moving image showing a predetermined action; an EMG stimulation generating device that generates EMG stimulation to stimulate the muscles of a subject; and an EMG control device that controls the EMG stimulation generating device to generate EMG stimulation corresponding to the predetermined action, wherein the EMG control device functions as an EMG control device that causes a delay so that the timing of the generation of EMG stimulation in the EMG stimulation generating device is delayed compared to the timing of the display of the moving image in the image display device. 【0009】 This invention makes it possible to improve the accuracy of motion reproduction by applying video and electromyographic stimulation to a target person. 【0010】This is a schematic block diagram showing the system configuration of the transfer system 100 of the present invention. This is a diagram showing an example configuration of the first embodiment of the electromyography information acquisition device 10. This is a diagram showing an example configuration of the second embodiment of the electromyography information acquisition device 10. This is a diagram showing an example configuration of the third embodiment of the electromyography information acquisition device 10. This is a diagram showing a schematic of the functional configuration of the delay information acquisition device 40. This is a diagram showing the concept of time difference. This is a diagram showing the concept of delay information. This is a flowchart showing a specific example of the operation flow of the transfer destination system 300. This is a diagram showing an example of the hardware configuration of the information processing device 90 applied to this embodiment. This is a diagram showing a first modified example of the transfer system 100. This is a diagram showing a second modified example of the transfer system 100. 【0011】 Figure 1 is a schematic block diagram showing the system configuration of the transcription system 100 of the present invention. The transcription system 100 comprises a source system 200 and a destination system 300. The source system 200 acquires information related to the actions that occur when a source subject performs a predetermined action (hereinafter referred to as "action-related information") and acquires electromyography information (hereinafter referred to as "source electromyography information") generated by the source subject's actions. Electromyography information is information that shows the time-series changes in the electrical potential (muscle action potential) generated in the muscles by the action. The source system 200 also captures a video image (hereinafter referred to as "source video image") of the source subject's actions. Based on the source electromyography information obtained by the source system 200, the destination system 300 acquires electromyography stimuli for the destination subject to perform the same predetermined action as the source subject and provides the electromyography stimuli to the destination subject. The destination system 300 also displays the source video image in a state that is visible to the destination subject. The recipient subject performs movements similar to those of the source subject by receiving electromyographic stimulation while viewing the source video. 【0012】The source system 200 and the destination system 300 are connected to each other via a network 70. The network 70 may be a wireless communication network or a wired communication network. The network 70 may be configured using, for example, the Internet or a local area network (LAN). The network 70 may be configured by combining multiple networks. 【0013】 The source system 200 includes an electromyography (EMG) information acquisition device 10, an imaging device 20, and a communication device 21. The EMG information acquisition device 10 acquires information related to the movements of the source subject (movement-related information) and outputs the EMG information of the source subject (source EMG information). The EMG information acquisition device 10 may be configured in any way as long as it can output source EMG information. 【0014】 Figures 2 to 4 show specific examples of the electromyography (EMG) information acquisition device 10. Specific examples of the configuration of the EMG information acquisition device 10 will be described below. Figure 2 shows an example of the configuration of the first embodiment of the EMG information acquisition device 10. The EMG information acquisition device 10 in the first embodiment includes an EMG sensor 11. The EMG sensor 11 is worn by the source subject. The EMG sensor 11 acquires electrical signals (EMG) generated by the movements of the source subject and acquires source EMG information based on the obtained electrical signals. In this way, in the first embodiment of the EMG information acquisition device 10, the source EMG information itself is acquired as movement-related information of the source subject. 【0015】Figure 3 shows an example configuration of the second embodiment of the electromyography (EMG) information acquisition device 10. The EMG information acquisition device 10 in the second embodiment includes a non-EMG sensor 12, a storage unit 13, and a control unit 14. The non-EMG sensor 12 acquires information other than EMG as motion-related information of the source subject. The non-EMG sensor 12 may be configured using, for example, a motion sensor that acquires information indicating changes in the position of various parts of the source subject's body. The non-EMG sensor 12 may be configured using, for example, an electroencephalogram (EEG) sensor that acquires brain waves generated when the source subject moves. The non-EMG sensor 12 may be configured using, for example, an imaging device that acquires moving images showing the movements of the source subject. The various sensors described above are just some specific examples of the non-EMG sensor 12. The non-EMG sensor 12 may be configured using any device that can acquire information other than EMG as motion-related information of the source subject. The non-EMG sensor 12 outputs the acquired information to the control unit 14. 【0016】 The storage unit 13 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 13 stores data used by the control unit 14. For example, the storage unit 13 functions as an electromyography (EMG) information determination model storage unit 131. The EMG information determination model storage unit 131 stores an EMG information determination model used by the EMG information acquisition unit 141 of the control unit 14 when processing. The EMG information determination model may be configured using, for example, a trained model acquired in advance by a training process. Specifically, the trained model may be obtained by performing a training process using training data that combines information during operation (explanatory variables) obtained using a sensor of the same type as the non-EMG sensor 12 and EMG information obtained during that operation (target variable). The EMG information determination model does not necessarily have to be acquired by a training process. The determination model may be configured using, for example, a lookup table that associates the explanatory variables and the target variable as described above, or it may be configured in other ways. 【0017】The control unit 14 is composed of a processor such as a CPU (Central Processing Unit) and memory (main memory). The control unit 14 functions as an electromyography information acquisition unit 141 when the processor executes a program. Note that all or part of the functions of the control unit 14 may be implemented using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, semiconductor memory devices (e.g., SSDs: Solid State Drives), and storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may be transmitted via a telecommunications line. 【0018】The control unit 14 may, for example, execute an application installed on its own device (electromyography information acquisition device 10). A specific example of such an application is an application provided to the electromyography information acquisition device 10 as a dedicated application for the transcription system 100. Another specific example of such an application is a web browser application. Such an application may be pre-installed on the electromyography information acquisition device 10, or it may be downloaded each time the electromyography information acquisition device 10 operates. For example, if it is implemented as a web browser application, the electromyography information acquisition device 10 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) when the electromyography information acquisition device 10 connects to a specific web server. The control unit 14 operates according to the program of the application being executed. The electromyography information acquisition unit 141 takes motion-related information obtained by the non-electromyography sensor 12 as input, acquires source electromyography information based on the electromyography information determination model, and outputs the acquired source electromyography information. 【0019】 Figure 4 shows an example configuration of the third embodiment of the electromyography (EMG) information acquisition device 10. The EMG information acquisition device 10 in the third embodiment includes a non-EMG sensor 12, a storage unit 13, and a control unit 14. The non-EMG sensor 12 in the EMG information acquisition device 10 of the third embodiment is the same as the non-EMG sensor 12 in the EMG information acquisition device 10 of the second embodiment, so its description is omitted. 【0020】 The memory unit 13 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The memory unit 13 functions as a motor information determination model storage unit 132 and an electromyography information determination model storage unit 133. 【0021】The motion information determination model storage unit 132 stores the motion information determination model used by the motion information acquisition unit 142 of the control unit 14 when processing. The motion information determination model may be configured using, for example, a trained model acquired in advance by a learning process. Specifically, the trained model may be obtained by performing a learning process using training data that combines information during movement (explanatory variables) obtained using a sensor of the same type as the non-myoelectric sensor 12, and information indicating the movement of the body of the subject performing the movement during that movement (hereinafter referred to as "motion information": target variable). The motion information is information that indicates the content of movement of a living organism (e.g., the human body). The motion information may be, for example, information indicating the movement of each joint of the living organism. More specifically, the motion information may be configured as information indicating the direction and amount of rotation of each joint, and the direction and amount of bending of each joint. The motion information determination model does not necessarily have to be acquired by a learning process. The determination model may be configured using, for example, a lookup table that associates the explanatory variables and the target variable as described above, or it may be configured in other ways. 【0022】 In the third embodiment, the electromyography (EMG) information determination model storage unit 133 stores an EMG information determination model used by the EMG information acquisition unit 143 of the control unit 14 when processing. The EMG information determination model may be configured using, for example, a trained model acquired in advance by a training process. Specifically, the trained model may be obtained by performing a training process using training data that combines motor information (explanatory variables) and EMG information (target variables) obtained when the motor information indicated by the motor information is performed. The EMG information determination model does not necessarily have to be acquired by a training process. The determination model may be configured using, for example, a lookup table that associates the explanatory variables and target variables as described above, or it may be configured in other ways. 【0023】The control unit 14 in the third embodiment is configured similarly to the control unit 14 in the second embodiment, except that it functions as an exercise information acquisition unit 142 and an electromyography information acquisition unit 143 instead of the electromyography information acquisition unit 141. The exercise information acquisition unit 142 takes motion-related information obtained by the non-electromyography sensor 12 as input, acquires exercise information based on an exercise information determination model, and outputs the acquired exercise information. The electromyography information acquisition unit 143 takes the exercise information obtained by the exercise information acquisition unit 142 as input, acquires source electromyography information based on an electromyography information determination model, and outputs the acquired source electromyography information. 【0024】 The above describes three specific configuration examples of the electromyography information acquisition device 10, from the first to the third embodiment. However, the configuration of the electromyography information acquisition device 10 is not limited to these three examples. 【0025】 Next, we return to Figure 1 and continue the explanation of the source system 200. The imaging device 20 is configured using a camera capable of capturing moving images. The imaging device 20 acquires source moving images by capturing images of the appearance of the source subject while they are performing an action. The communication device 21 is configured using a communication interface capable of communication. The communication device 21 transmits the source electromyography information and source moving images obtained by the electromyography information acquisition device 10 and the imaging device 20, respectively, when the source subject is performing the same action, to other devices. In this embodiment, the communication device 21 transmits the source electromyography information and source moving images to the destination system 300. 【0026】Next, the destination system 300 will be described. The destination system 300 includes a communication device 30, an image display device 31, an electromyography (EMG) stimulation control device 32, an EMG stimulation generation device 33, a sensor 34, and a delay information acquisition device 40. The communication device 30 is configured using a communication interface. The communication device 30 receives source EMG information and source video from other devices. In this embodiment, the communication device 30 receives source EMG information and source video transmitted from the source system 200. The communication device 30 outputs the received source EMG information and source video. The image display device 31 is configured using an image display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro Luminescence) display. The image display device 31 displays, for example, the source video received by the communication device 30. 【0027】 The electromyography (EMG) stimulation control device 32 is a device that provides electromyography (EMG) stimulation to a living organism. The EMG stimulation control device 32 may be configured, for example, as a device that controls EMS. The EMG stimulation control device 32 receives source EMG information received by the communication device 30, acquires a control signal for the EMG stimulation generation device 33, and outputs a control signal. The EMG stimulation control device 32 acquires a control signal to generate an EMG stimulus that will produce EMG information similar to the input source EMG information from the target subject. The EMG stimulation control device 32 outputs a control signal synchronized with the content of the source video displayed on the image display device 31. That is, at the moment the image display device 31 displays the source video, the EMG stimulation control device 32 outputs a control signal so that an EMG stimulus to reproduce the source EMG information obtained when the displayed source video was captured is applied to the living organism from the EMG stimulation generation device 33. It is desirable that such synchronization be performed in advance in the target system 300. When the electromyography stimulation control device 32 receives delay information from the delay information acquisition device 40, it generates a delay of the time indicated by the delay information from the input timing of the source electromyography information and outputs a control signal to the electromyography stimulation generation device 33. Therefore, when delay information is input, the synchronization is shifted by the length of time indicated by the delay information. 【0028】The electromyography (EMG) stimulation generator 33 is configured using a device that delivers EMG stimulation to a living organism. The EMG stimulation generator 33 may be configured using, for example, EMS. The EMG stimulation generator 33 may be worn by, for example, the recipient of the transfer. 【0029】 Sensor 34 acquires motion-related information of the target subject. Sensor 34 may be configured using, for example, an electromyography sensor or a non-electromyography sensor. Sensor 34 outputs the acquired motion-related information of the target subject to the delayed information acquisition device 40. 【0030】 Figure 5 is a schematic diagram of the functional configuration of the delay information acquisition device 40. The delay information acquisition device 40 comprises a storage unit 41 and a control unit 42. The storage unit 41 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 41 stores data used by the control unit 42. For example, the storage unit 41 functions as a first reference information determination model storage unit 411 and a second reference information determination model storage unit 412. 【0031】 The first reference information determination model storage unit 411 stores the first reference information determination model used by the first reference information acquisition unit 421 of the control unit 42 when processing. The first reference information determination model may be configured using, for example, a trained model acquired in advance by a training process. Specifically, the trained model may be obtained by performing a training process using training data that combines motion-related information (explanatory variables) obtained using a sensor of the same type as the sensor 34 and the first reference information (target variable) obtained during that motion. The first reference information can be any information that can be obtained as time-series information. For example, the first reference information may be configured using electromyography information or using motion information. The first motion information determination model does not necessarily have to be acquired by a training process. The determination model may be configured using, for example, a lookup table that associates the explanatory variables and the target variable as described above, or it may be configured in other ways. 【0032】The second reference information determination model storage unit 412 stores the second reference information determination model used by the second reference information acquisition unit 422 of the control unit 42 when processing. The second reference information determination model may be configured using, for example, a trained model acquired in advance by a training process. Specifically, the trained model may be obtained by performing a training process using training data that combines images of body parts of the source subject in the source video (explanatory variables) and second reference information (target variables) obtained when the body parts in the images are moving. The second reference information in the second reference information determination model is of the same type as the first reference information in the first reference information determination model described above. For example, if the first reference information is configured using electromyography information, the second reference information will also be configured using electromyography information. For example, if the first reference information is configured using motor information, the second reference information will also be configured using motor information. The second motor information determination model does not necessarily need to be acquired by a training process. The determination model may be configured using, for example, a lookup table that associates the explanatory variables and target variables as described above, or it may be configured in other ways. 【0033】 The control unit 42 is configured using a processor such as a CPU and memory. The control unit 42 functions as a first reference information acquisition unit 421, a second reference information acquisition unit 422, a time difference acquisition unit 423, and a timing adjustment unit 434 when the processor executes a program. Note that all or part of each function of the control unit 42 may be implemented using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor storage devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor storage devices built into computer systems. The above program may be transmitted via a telecommunications line. 【0034】The control unit 42 may, for example, execute an application installed on its own device (delay information acquisition device 40). A specific example of such an application is an application provided to the delay information acquisition device 40 as a dedicated application for the transcription system 100. Another specific example of such an application is a web browser application. Such an application may be pre-installed on the delay information acquisition device 40, or it may be downloaded each time the delay information acquisition device 40 operates. For example, if it is implemented as a web browser application, the delay information acquisition device 40 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) in response to the delay information acquisition device 40 connecting to a specific web server. The control unit 42 operates according to the program of the application being executed. 【0035】 The first reference information acquisition unit 421 takes the operation-related information obtained by the sensor 34 as input, acquires first reference information based on the first reference judgment model, and outputs the acquired first reference information to the time-difference acquisition unit 423. 【0036】 The second reference information acquisition unit 422 takes the source video received by the communication device 30 as input and extracts images of the human body contained within the video. The second reference information acquisition unit 422 extracts images of the human body using, for example, template matching or a trained model. At this time, the second reference information acquisition unit 422 may extract multiple images for each specific part of the human body. For example, the second reference information acquisition unit 422 may extract an image for each finger. Any existing technology may be applied to extract images of the human body from the video. Using the extracted images of the human body, the second reference information acquisition unit 422 acquires second reference information based on the second reference judgment model and outputs the acquired second reference information to the time-delay acquisition unit 423. 【0037】The time difference acquisition unit 423 takes the first reference information acquired by the first reference information acquisition unit 421 and the second reference information acquired by the second reference information acquisition unit 422 as input, acquires the time difference, and outputs the acquired time difference. Figure 6 is a diagram illustrating the concept of time difference. In the source system 200, the timing 81 when the source subject performs an action A and the timing 82 when the electromyographic information generated in response to that action A 81 is obtained are basically the same timing, except for the delay that occurs in the equipment of the source system 200. 【0038】 In contrast, there is a time difference between the timing 83 when the source video of the source subject performing a certain action A is displayed on the image display device 31, and the timing 84 when the recipient subject performs action A after viewing the source video of action A. There are various reasons for this, but one reason is that it takes time from when the recipient subject sees the action video A until they actually try to move their body. In Figure 6, this time difference is represented as Δt1. The time difference acquisition unit 423 acquires the timing at which the same action was performed using the first reference information and the second reference information, respectively. The time difference acquisition unit 423 then acquires the difference in these timings as a time difference. 【0039】The delay information acquisition unit 424 takes a time difference as input, acquires delay information, and outputs the acquired delay information. The delay information is information indicating the amount by which the output timing of the control signal for electromyography stimulation synchronized with the source video image is delayed. The delay information acquisition unit 424 acquires delay information using a predetermined algorithm based on the input time difference. The delay information acquisition unit 424 may, for example, acquire the time difference itself as delay information. The delay information acquisition unit 424 may, for example, perform a predetermined operation on the time difference and acquire delay information as the result of the operation. Specific examples of such operations include adding or subtracting a predetermined constant, multiplying or dividing by a predetermined constant, or a combination of these. The delay information acquisition unit 424 may also acquire delay information using previously input time difference values ​​in addition to the input time difference. The delay information acquisition unit 424 may output delay information using an initial value from the start of processing until the time difference is obtained. 【0040】 Figure 7 illustrates the concept of delay information. In Figure 7, this delay is represented as Δt2. As described above, there is a time lag between when the recipient sees the action video A and when they actually attempt to move their body. Therefore, the electromyographic stimulation required to perform action A is more effective when it is given to the recipient at a timing 84 after a delay Δt2 has occurred from the display of action video A, rather than being given to the recipient at the exact same timing 83 as the display of action video A. This allows the recipient to perform action A with greater accuracy. Therefore, by delaying the timing of the electromyographic stimulation output by the electromyographic stimulation generator 33 based on the delay information acquired by the delay information acquisition unit 424, it is possible to increase the accuracy of the reproduction of the action by the recipient. 【0041】Figure 8 is a flowchart illustrating a specific example of the operation flow of the transfer destination system 300. The first reference information determination unit 431 of the delay information acquisition device 40 receives the output (operation-related information) of the sensor 34 as input and acquires first reference information corresponding to the operation-related information (step S101). The first reference information determination unit 431 outputs the acquired first reference information to the time-difference acquisition unit 423. The second reference information determination unit 432 receives the received source video image as input and acquires second reference information (step S102). The second reference information determination unit 432 outputs the acquired second reference information to the time-difference acquisition unit 423. 【0042】 The time difference acquisition unit 423 receives the first reference information and the second reference information as input and acquires the time difference between the reference information (step S203). For example, the time difference acquisition unit 423 may determine the timing at which the same action occurred in the first reference information and the second reference information, and acquire the time between those timings as a time difference. The time difference acquisition unit 423 outputs the acquired time difference to the delay information acquisition unit 424. The delay information acquisition unit 424 receives the time difference as input and acquires delay information based on a predetermined algorithm (step S204). The delay information acquisition unit 424 outputs a control signal to the electromyography stimulation generator 33 so that the electromyography stimulation is output from the electromyography stimulation generator 33 with a delay of the time indicated by the delay information from the display timing of the source video (step S205). 【0043】 Figure 9 is a schematic diagram of an example hardware configuration of an information processing device 90 applied to this embodiment. The information processing device 90 comprises a processor 91, a main memory 92, a communication interface 93, an auxiliary storage device 94, an input / output interface 95, and an internal bus 96. The processor 91, the main memory 92, the communication interface 93, the auxiliary storage device 94, and the input / output interface 95 are connected to each other via the internal bus 96 so as to be able to communicate with each other. The information processing device 90 may be applied, for example, to an electromyography information acquisition device 10 and a delay information acquisition device 40. In this case, for example, the storage unit 13 and the storage unit 41 may be configured using the auxiliary storage device 94. Also, the control unit 14 and the control unit 42 may be configured using the processor 91 and the main memory 92. 【0044】 According to the transfer system 100 configured as described above, the source moving image obtained for the source subject is reproduced and displayed on the image display device 31 of the destination system 300, and the myoelectric stimulation corresponding to the source myoelectric information is applied to the destination subject by the myoelectric stimulation generation device 33. At that time, a delay is given between the timing of the source moving image and the timing when the myoelectric stimulation is applied. Therefore, it is possible to improve the accuracy of reproducing the operation by applying the video and the myoelectric stimulation to the person at the destination. 【0045】 Further, in the transfer system 100, the amount of delay given to the timing of the myoelectric stimulation is dynamically determined according to the individual abilities and states of the destination subjects. Therefore, it is possible to further improve the accuracy of reproducing the operation by applying the video and the myoelectric stimulation to the person at the destination. 【0046】 (Modified Example) FIG. 10 is a diagram showing a first modified example of the transfer system 100. In the first modified example, the communication device 21 of the source system 200 transmits the source moving image and the source myoelectric information to the transfer information storage device 50 instead of the destination system 300. The transfer information storage device 50 is configured using an information processing device and a storage device such as a magnetic hard disk device or a semiconductor storage device. When the transfer information storage device 50 receives the source moving image and the source myoelectric information from the source system 200, it records the received information. When the transfer information storage device 50 receives a request from the destination system 300, it reads out the source moving image and the source myoelectric information corresponding to the received request from the storage device and transmits them to the destination system 300 that is the request source. 【0047】The transcription information storage device 50 may store the source moving image and the source myoelectric information for each type of operation (e.g., a combination of an instrument name and a music piece name, a combination of a sport name and an operation name, etc.) when the source moving image and the source myoelectric information are obtained. At this time, the transcription information storage device 50 may store the source moving image and the source myoelectric information in association with the identification information (e.g., affiliation, name, etc.) of the source target person. When the transcription destination system 300 receives information specifying the source moving image and the source myoelectric information from a user such as the transcription destination target person, it may send a request for the specified source moving image and the source myoelectric information to the transcription information storage device 50. In this case, when the transcription destination system 300 receives the source moving image and the source myoelectric information from the transcription information storage device 50, it operates based on these information as described above. 【0048】 FIG. 11 is a diagram showing a second modification example of the transcription system 100. In the second modification example, the source system 200 includes a transcription information storage device 23 instead of the communication device 21, and the transcription destination system 300 includes a transcription information storage device 35 instead of the communication device 30. The transcription information storage device 23 of the source system 200 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The transcription information storage device 23 stores the source moving image and the source myoelectric information in association with each other. At this time, the transcription information storage device 23 may store the source moving image and the source myoelectric information in association with the type of operation and the identification information of the source target person as described above. The information stored in the transcription information storage device 23 is transferred to the transcription information storage device 35 included in the transcription destination system 300. For example, the information may be transferred by communicating via a network, or the transfer may be performed via a recording medium such as a USB memory or a DVD-ROM. The transcription destination system 300 operates based on the source moving image and the source myoelectric information stored in the transcription information storage device 35. 【0049】The electromyography (EMG) information acquisition device 10 may be implemented using multiple information processing devices. For example, the EMG information acquisition device 10 may be implemented using a device such as a cloud. For example, in the EMG information acquisition device 10, the storage unit 13 and the control unit 14 may be implemented in different information processing devices. For example, the control unit 14 of the EMG information acquisition device 10 may be distributed and implemented in multiple information processing devices. 【0050】 The delay information acquisition device 40 may be implemented using multiple information processing devices. For example, the delay information acquisition device 40 may be implemented using a device such as a cloud. For example, in the delay information acquisition device 40, the storage unit 41 and the control unit 42 may be implemented in different information processing devices. For example, the control unit 42 of the delay information acquisition device 40 may be distributed and implemented in multiple information processing devices. 【0051】 The transfer destination system 300 may further include an imaging device. The imaging device of the transfer destination system 300 captures images of the transfer destination subject performing actions using the transfer destination system 300. The imaging device of the transfer destination system 300 may capture images of the transfer destination subject with the same or similar viewpoint parameters (viewpoint position, viewpoint direction, field of view, etc.) as the imaging device 20 of the source system 200 captures images of the source subject. In this case, when the image display device 31 displays the source video, it may superimpose and display the image captured by the imaging device of the transfer destination system 300. The source video may be processed and displayed on the image display device 31. For example, the source video may be processed and displayed so as to be represented using computer graphics. The image display device 31 may also display video images expressed in other forms, as long as they show the same action as the source video being performed at the same timing. For example, the image display device 31 may display video images in which icons, illustrations, or text representing the action are displayed on the screen at the timing when the action is performed. Such moving images may be generated in the source system 200, in the destination system 300, or using other information processing equipment. 【0052】The transfer destination system 300 may be configured without a sensor 34 and a delay information acquisition device 40. In this case, the electromyography stimulation control device 32 may operate using predetermined delay information. Such predetermined delay information may be one or more. If multiple delay information is predetermined, for example, the system may be configured so that the user of the transfer destination system 300 can select which delay information to use, or it may be predetermined which delay information to use for the source video and source electromyography information used in the transfer destination system 300. 【0053】 Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. 【0054】 100...Transfer system, 10...Electromyography information acquisition device, 11...Electromyography sensor, 12...Non-electromyography sensor, 13...Memory unit, 131...Electromyography information judgment model memory unit, 132...Motion information judgment model memory unit 132, 133...Electromyography information judgment model memory unit, 14...Control unit, 141...Electromyography information acquisition unit, 142...Motion information acquisition unit, 143...Electromyography information acquisition unit, 20...Imaging device, 21...Communication device, 200...Transfer source system, 30...Communication device, 31...Image display device, 32...Electromyography stimulation control device, 33...Electromyography stimulation generation device, 34...Sensor, 300...Transfer destination system (control system), 40...Delay information acquisition device, 41...Memory unit, 411...First reference information judgment model memory unit, 412...Second reference information judgment model memory unit, 42...Control unit, 421...First reference information acquisition unit, 422...Second reference information acquisition unit, 423...Time difference acquisition unit, 424...Delay information acquisition unit

Claims

1. A control system comprising: an image display device that displays a moving image showing a predetermined action; an electromyographic stimulation (EMG) generation device that generates EMG stimulation to stimulate the muscles of a subject; and an EMG control device that controls the EMG generation device to generate EMG stimulation corresponding to the predetermined action, wherein the EMG control device causes a delay such that the timing of the generation of EMG stimulation in the EMG generation device is delayed compared to the timing of the display of the moving image in the image display device.

2. The control system according to claim 1, further comprising: a sensor that acquires motion-related information which is information related to the movements of the subject; and a delay information acquisition device that acquires information indicating the delay based on the motion-related information, wherein the electromyography stimulation control device generates the delay acquired by the delay information acquisition device.

3. A control method comprising: an image display step of displaying a moving image showing a predetermined action; an electromyographic stimulation generation step of generating electromyographic stimulation that stimulates the muscles of a subject; and an electromyographic stimulation control step of controlling the timing of the execution of the electromyographic stimulation generation step to generate electromyographic stimulation corresponding to the predetermined action, wherein in the electromyographic stimulation control step, a delay is introduced such that the timing of the generation of the electromyographic stimulation is delayed compared to the timing of the display of the moving image.

4. An information system comprising: an image display device that displays a moving image showing a predetermined action; an electromyographic stimulation (EMG) generation device that generates EMG stimulation to stimulate the muscles of a subject; and an EMG control device that controls the EMG generation device to generate EMG stimulation corresponding to the predetermined action, wherein the EMG control device functions as an EMG control device that causes a delay so that the timing of the generation of EMG stimulation in the EMG generation device is delayed compared to the timing of the display of the moving image in the image display device.

Images