Information processing system

Abstract

Disclosed is an information processing system (S) including an operating-side device (1) and an operated-side device (2), the operating-side device (1) and the operated-side device (2) being capable of communicating with each other over a network, wherein the operating-side device (1) accepts input of torque information about the operating-side device (1) on the basis of an operation at the operating-side device (1) and transmits the torque information to the operated-side device (2), the operated-side device (2) calculates a target position and an intensity of action by the operated-side device (2) on the basis of the torque information and engages in action in accordance with the calculation results, the operated-side device (2) detects an impact or resisting force upon contact with an obstacle during the action as a reaction force of the action and transmits reaction force information to the operating-side device (1), and the operating-side device (1) calculates haptic information, including a target position and an intensity of action by the operating-side device (1), on the basis of the reaction force information and engages in action in accordance with the calculation result.

Description

Information Processing Systems

[0001] The present disclosure relates to an information processing system.

[0002] Conventionally, remote control systems that operate robot arms remotely via a network have been known (see, for example, Non-Patent Document 1). A typical remote control mechanism involves providing the same operation target for a leader (operating side) and a follower (operated side), and remote control is performed by synchronizing positions based on position information. Generally, by operating the leader, the follower moves in accordance with the leader's movements based on the leader's position information. Here, consider a system in which the follower provides feedback of tactile information to the leader, and the leader operates while checking the tactile sensation (resistance, etc.). In this case, control based on position information is performed bidirectionally between the leader and follower, and operation and tactile feedback are generally achieved through position synchronization.

[0003] Motionlib, Inc., "AbcCore, an IC chip that realizes Real Haptics (registered trademark)," [Online], 2022, [December 6, 2023], Internet <URL: https: / / www.motionlib.com / product / abc-core / >

[0004] When position synchronization is performed for both the leader and follower, and the feedback from the follower to the leader is large, the leader can sense even the slightest tactile information generated by the follower. However, the leader's operation becomes stiff and heavy due to the position synchronization of the follower's current position. On the other hand, when the feedback from the follower to the leader is small, the leader can operate the follower with smooth movements. However, since the tactile information generated by the follower is reduced and fed back, there is a disadvantage that the leader has difficulty sensing the slightest tactile information. It is common to tune the weighting coefficient of the position synchronization feedback depending on the scene, but there are cases where it is not possible to achieve both a smooth operation feel and sensitive tactile feedback.

[0005] In view of the above circumstances, an object of the present disclosure is to achieve both a smooth operation feel and sensitive tactile feedback in remote control.

[0006] An information processing system according to one embodiment is an information processing system including an operating side device and an operated side device, wherein the operating side device and the operated side device are capable of communicating with each other via a network, the operating side device receives input of torque information from an operation on the operating side device and transmits the torque information to the operated side device, the operated side device calculates a target position and strength of an action of the operated side device from the torque information and operates in accordance with the calculation results, the operated side device detects an impact or resistance force when coming into contact with an obstacle during the action as a reaction force to the action and transmits reaction force information to the operating side device, and the operating side device calculates force-tactile information including a target position and strength of an action of the operating side device from the reaction force information and operates in accordance with the calculation results.

[0007] According to the present disclosure, it is possible to achieve both a smooth operation feel and sensitive tactile feedback during remote operation.

[0008] 1 is a block diagram showing a configuration of an information processing system according to an embodiment; FIG. 2 is a block diagram showing a configuration of an operating side device; FIG. 3 is a diagram showing a configuration of an operated side device; FIG. 4 is a diagram showing a first example of a combination of an operating side device and an operated side device; FIG. 5 is a diagram showing a second example of a combination of an operating side device and an operated side device; and FIG. 6 is a sequence diagram showing information processing by the information processing system.

[0009] An embodiment will be described below with reference to the drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals. In the description of this embodiment, the description of identical or corresponding parts will be omitted or simplified as appropriate.

[0010] 1 is a schematic diagram of an information processing system S according to this embodiment. The information processing system S includes an operating device 1 and an operated device 2 that can communicate with each other via a network NW. The operating device 1 is also referred to as a leader device. The operated device 2 is also referred to as a follower device. The network NW includes, for example, a mobile communication network, a fixed communication network, or the Internet.

[0011] The information processing system S is a remote control system in which an operating device 1 remotely operates an operated device 2 via a network NW. The operated device 2 is, for example, a robot. To perform precise operation, it is necessary to feed back the tactile sensation of the operated device 2 to the operating device 1, and for the operator of the operating device 1 to operate the operated device 2 while checking the tactile sensation of the operated device 2 in real time. The impact or resistance force when the operated device 2 grabs or hits an object is fed back to the operating device 1 in real time. The operator of the operating device 1 can operate the operated device 2 while receiving the feedback.

[0012] The network NW may include, for example, the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. The network may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network may be, for example, an ad-hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0013] 1 shows two operating side devices 1 and two operated side devices 2. However, the number of operating side devices 1 and two operated side devices 2 is not limited to this.

[0014] 2 illustrates the internal configuration of the operating-side device 1. The operating-side device 1 includes an operating-side interface 11, a position information detection unit 12, a torque information detection unit 13, a motion calculation unit 14, a transmission unit 15, a reception unit 16, and a communication port 17. The components of the operating-side device 1 are connected to each other so that they can communicate with each other.

[0015] The operating-side interface 11 is a physical interface of the operating-side device 1 .

[0016] The position information detection unit 12 detects position information of the operating-side device 1 (e.g., the operating-side interface 11). The position information detection unit 12 may be, for example, a sensor. The torque information detection unit 13 detects torque information of the operating-side device 1 (e.g., the operating-side interface 11). The torque information detection unit 13 may be, for example, a sensor.

[0017] The motion calculation unit 14 calculates the target position or intensity of the motion by the operating side device 1 .

[0018] The transmitting section 15 transmits information relating to the operating side device 1 to the operated side device 2. The receiving section 16 receives information relating to the operated side device 2. The communication port 17 is an interface used for transmission and reception.

[0019] 3 illustrates the internal configuration of the operated side device 2. The operated side device 2 includes an operated side interface 21, a reaction force information detection unit 22, a motion calculation unit 23, a position correction unit 24, a transmission unit 25, a reception unit 26, and a communication port 27. The components of the operated side device 2 are connected to each other so that they can communicate with each other.

[0020] The operated side interface 21 is a physical interface of the operated side device 2 .

[0021] The reaction force information detection section 22 detects reaction force information of the operated side device 2. The reaction force information detection section 22 may be, for example, a sensor.

[0022] The motion calculation section 23 calculates the target position and strength of the motion by the operated side device 2. The position correction section 24 corrects the position of the operated side interface 21.

[0023] The transmitting section 25 transmits information relating to the operated-side device 2 to the operating-side device 1. The receiving section 26 receives information relating to the operating-side device 1. The communication port 27 is an interface used for transmitting and receiving information.

[0024] The operating side device 1 and the operated side device 2 may include a storage unit. The storage unit includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. A flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. A magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit functions as, for example, a main storage device, an auxiliary storage device, or a cache memory.

[0025] The functions of the motion calculation unit 14, the transmission unit 15, the reception unit 16, the motion calculation unit 23, the position correction unit 24, the transmission unit 25, and the reception unit 26 are performed by a control unit. The control unit includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit may include one or more dedicated processors specialized for specific operations. Instead of including a processor, the control unit may include one or more dedicated circuits. The dedicated circuits may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit may include an ECU (Electronic Control Unit). The control unit controls the communication unit to send and receive any information.

[0026] 4 and 5 show examples of combinations of the operating-side device 1 and the operated-side device 2. The operating-side device 1 and the operated-side device 2 may have the same shape or similar shapes. The movement amount and torque due to the position information control and the torque information control may be the same between the operating-side device 1 and the operated-side device 2, or may be converted to different magnitudes. As shown in FIG. 4, the size of the operated-side device 2 is 10 times the size of the operating-side device 1, and the operated-side device 2 operates with a torque 10 times the torque applied to the operating-side device 1. The reaction force generated in the operated-side device 2 is fed back to the operating-side device 1 at one-tenth the magnitude.

[0027] The operating device 1 may have a different shape from the operated device 2. For example, as shown in FIG. 5 , the operating device 1 may have a shape similar to that of a game console controller, and the operated device 2 may have a shape similar to that of a robot. In this case, tactile information of different natures may be associated with each other, such as the tilt of a stick on a game console and the movement of a robot arm. Different actions may be associated with each other on the operating device 1 and the operated device 2, such as feeding back a rotational action in response to a linear action.

[0028] Each process executed by the information processing system S will be described in detail.

[0029] [First embodiment] The operating-side device 1 is operated by an operator. Current position information and current torque information of the operating-side device 1 are input to the operating-side interface 11 through operation on the operating-side device 1. Upon receiving the input, the operating-side device 1 transmits the input position information and torque information to the operated-side device 2 periodically or irregularly. The position information may include information indicating tilt or attitude, etc. The torque information is calculated by the motion calculation unit 14 using, for example, the acceleration of the operation on the operating-side device 1 or the current value generated in the motor of the operating-side device 1. The torque information is an external force when the operating-side device 1 is manually operated, and may have directionality and strength.

[0030] The operated-side device 2 calculates the target position and strength of the operation of the operated-side device 2 from the position information and torque information of the operating-side device 1, and operates according to the calculation results, following the movement of the operating-side device 1. The positions of the operating-side device 1 and the operated-side device 2 are synchronized by position information control. The operated-side device 2 detects the impact or resistance force when it comes into contact with an obstacle during operation as a reaction force to the operation. The reaction force is calculated, for example, from the current value generated in the motor of the operated-side device 2, and may have directionality and strength. The operated-side device 2 transmits reaction force information to the operating-side device 1 periodically or irregularly while operating.

[0031] The operating side device 1 calculates haptic information including the target position and strength of the operation of the operating side device 1 from the reaction force information received from the operated side device 2, and operates according to the calculation result. By control using torque information, the operation of the operating side device 1 is controlled only when a reaction force is generated. When no reaction force is generated, the operating side device 1 realizes smooth operation without any resistance.

[0032] Second Embodiment In a second embodiment, the position information of the operating side device 1 is not used. Contents common to the first embodiment will be omitted as appropriate.

[0033] Current torque information is input to the operating interface 11 by operating the operating side device 1. The operating side device 1 transmits the torque information to the operated side device 2 periodically or irregularly.

[0034] The operated-side device 2 calculates the target position and strength of the operation of the operated-side device 2 from the torque information of the operating-side device 1, and operates in accordance with the calculation results.

[0035] In the second embodiment, the operation of the operated-side device 2 is controlled using torque information only when a reaction force is generated in the operating-side device 1. The operation of the operating-side device 1 is controlled using torque information only when a reaction force is generated in the operated-side device 2.

[0036] [Third Embodiment] In the third embodiment, in addition to the control using torque information in the second embodiment, position control is performed. Specifically, when the position of the operating side device 1 and the position of the operated side device 2 are misaligned due to the control using torque information, position correction is performed. Contents common to the second embodiment will be omitted as appropriate.

[0037] In the second embodiment, the presence or absence of a difference between the position of the operating side device 1 and the position of the operated side device 2 is determined periodically or irregularly using the position information of the operating side device 1 and the operated side device 2. The determination of the presence or absence of a difference is performed by the operated side device 2, but may alternatively be performed by the operating side device 1.

[0038] When the operated side device 2 determines that a difference has occurred between the position of the operating side device 1 (e.g., the operating side interface 11) and the position of the operated side device 2 (e.g., the operated side interface 21), the position correction unit 24 corrects the position of the operated side device 2 (e.g., the operated side interface 21) using the position information of the operating side device 1 (e.g., the operating side interface 11).

[0039] As an alternative example, the operating side device 1 may correct the position of the operating side device 1 using the position information of the operated side device 2 .

[0040] 6 illustrates a process executed in the information processing system S. This process may be executed periodically.

[0041] In S1, the operating-side device 1 accepts input of torque information of the operating-side device 1 from operation on the operating-side device 1. As an additional example, position information of the operating-side device 1 may be input. In S2, the operating-side device 1 transmits the torque information to the operated-side device 2. As an additional example, position information of the operating-side device 1 may be transmitted.

[0042] In S3, the operated side device 2 calculates the target position and strength of the operation of the operated side device 2 from the torque information. In S4, the operated side device 2 operates according to the calculation result. In S5, the operated side device 2 detects the impact or resistance force when it comes into contact with an obstacle during operation as a reaction force to the operation. In S6, the operated side device 2 transmits the reaction force information to the operating side device 1. As an additional example, the operated side device 2 may determine whether there is a difference between the position of the operating side device 1 and the position of the operated side device 2. If the operated side device 2 determines that a difference has occurred, it may correct the position of the operated side device 2 using the position information of the operating side device 1.

[0043] In S7, the operating side device 1 calculates haptic information including the target position and strength of the operation of the operating side device 1 from the reaction force information, and operates in accordance with the calculation result.

[0044] As described above, according to this embodiment, the operating-side device 1 receives input of torque information for the operating-side device 1 from an operation on the operating-side device 1 and transmits the torque information to the operated-side device 2. The operated-side device 2 calculates the target position and strength of the operation of the operated-side device 2 from the torque information and operates according to the calculation result. The operated-side device 2 detects the impact or resistance force when it comes into contact with an obstacle during operation as a reaction force to the operation and transmits reaction force information to the operating-side device 1. The operating-side device 1 calculates haptic information including the target position and strength of the operation of the operating-side device 1 from the reaction force information and operates according to the calculation result. With this configuration, the information processing system S can achieve both a smooth operation feel and sensitive haptic feedback during remote operation. Furthermore, when no reaction force is generated in the operated-side device 2, the information processing system S can achieve smooth operation without any resistance in the operating-side device 1.

[0045] Furthermore, according to this embodiment, the operating-side device 1 accepts input of position information of the operating-side device 1 in addition to torque information, and transmits the position information in addition to the torque information to the operated-side device 2. The operated-side device 2 calculates the target position and strength of the operation of the operated-side device 2 from the torque information and position information. This configuration enables the information processing system S to achieve even smoother operation feel and more sensitive tactile feedback.

[0046] Furthermore, according to this embodiment, either the operating side device 1 or the operated side device 2 determines whether there is a difference between the position of the operating side device 1 and the position of the operated side device 2, and if it determines that a difference has occurred, corrects the position of either the operating side device 1 or the operated side device 2 using the position information of the other of the operating side device 1 or the operated side device 2. This configuration enables the information processing system S to achieve even smoother operation feel and more sensitive tactile feedback.

[0047] Furthermore, according to this embodiment, the torque information is calculated from the acceleration of the operation on the operating side device 1 or the current value generated in the motor of the operating side device 1, and the reaction force is calculated from the current value generated in the motor of the operated side device 2. With this configuration, the information processing system S can achieve even more sensitive tactile feedback.

[0048] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagram may be integrated, or one block may be divided. Two or more steps shown in the flowchart may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure.

[0049] Each device constituting the information processing system S of this embodiment can also be realized by a computer and a program. Furthermore, in the above-described embodiment, a program that executes all or part of the functions or processes of each device can be recorded on a computer-readable recording medium or provided via a network. Computer-readable recording media include non-transitory computer-readable media, such as magnetic recording devices, optical disks, magneto-optical recording media, or semiconductor memories. The program can be distributed, for example, by selling, transferring, or lending portable recording media, such as Secure Digital (SD) cards, Digital Versatile Discs (DVDs), or Compact Disc Read-Only Memory (CD-ROMs), on which the program is recorded. The program can also be distributed by storing the program in the storage of a server and transmitting it from the server to another computer via a network. The program can also be provided as a program product. The present disclosure can also be realized as a program executable by a processor. Some or all of the functions of each device can be implemented by programmable circuits or dedicated circuits. Some or all of the functions of each device can be implemented by hardware.

[0050] The following supplementary notes further disclose the above-described embodiments.

[0051] (Additional Item 1) An information processing system including an operating side device and an operated side device, wherein the operating side device and the operated side device are capable of communicating with each other via a network, the operating side device accepts input of torque information of the operating side device from an operation on the operating side device and transmits the torque information to the operated side device, the operated side device calculates a target position and strength of an action of the operated side device from the torque information and operates in accordance with the calculation results, the operated side device detects an impact or resistance force when coming into contact with an obstacle during the action as a reaction force to the action and transmits reaction force information to the operating side device, and the operating side device calculates force tactile information including a target position and strength of an action of the operating side device from the reaction force information and operates in accordance with the calculation results. (Supplementary Item 2) In the information processing system described in Supplementary Item 1, the operating side device accepts input of position information of the operating side device in addition to the torque information, and transmits the position information in addition to the torque information to the operated side device, and the operated side device calculates the target position and the strength of the motion of the operated side device from the torque information and the position information. (Supplementary Item 3) In the information processing system described in Supplementary Item 1, one of the operating side device or the operated side device determines whether there is a difference between the position of the operating side device and the position of the operated side device, and if it determines that there is a difference, corrects the position of the one of the operating side device or the operated side device using position information of the other of the operating side device or the operated side device. (Supplementary Item 4) In the information processing system described in any one of Supplementary Items 1 to 3, the torque information is calculated from the acceleration of the operation on the operating side device or the current value generated in the motor of the operating side device, and the reaction force is calculated from the current value generated in the motor of the operated side device.

[0052] S Information Processing System

Claims

1. An information processing system including an operating side device and an operated side device, wherein the operating side device and the operated side device can communicate with each other via a network, the operating side device receives an input of torque information of the operating side device from an operation on the operating side device and transmits the torque information to the operated side device, the operated side device calculates a target position and intensity of the operation of the operated side device from the torque information and operates according to the calculation result, the operated side device detects an impact or resistance force when contacting an obstacle during the operation as a reaction force of the operation and transmits reaction force information to the operating side device, and the operating side device calculates force feedback information including a target position and intensity of the operation of the operating side device from the reaction force information and operates according to the calculation result.

2. The information processing system according to claim 1, wherein the operating side device receives an input of position information of the operating side device in addition to the torque information and transmits the position information to the operated side device in addition to the torque information, and the operated side device calculates the target position and the intensity of the operation of the operated side device from the torque information and the position information.

3. The information processing system according to claim 1, wherein one of the operating side device or the operated side device determines whether there is a difference between the position of the operating side device and the position of the operated side device, and when it is determined that the difference has occurred, corrects the position of one of the operating side device or the operated side device using the position information of the other of the operating side device or the operated side device.

4. The information processing system according to claim 1, wherein the torque information is calculated from an acceleration of an operation on the operating side device or a current value generated in a motor of the operating side device, and the reaction force is calculated from a current value generated in a motor of the operated side device.

Images