Movement control device and method

Abstract

A movement control device according to one embodiment comprises: a division unit that acquires information indicating a trajectory of movement of a human body along a third surface located between a first surface and a second surface that is at a different angle from the angle of the first surface, divides the trajectory indicated by the acquired information into a trajectory of movement along the first surface and a trajectory of movement along the second surface, and outputs the divided trajectories; and control unit that controls the movement of a first object along the trajectory of movement along the first surface that is outputted by the division unit when movement control of the first object that simulates the body movement is enabled, and controls the movement of a second object along the trajectory of movement along the second surface that is outputted by the division unit when movement control of the second object that simulates the body movement is enabled.

Description

Motion control device and method

[0001] FIELD OF THE INVENTION Embodiments of the present invention relate to motion control devices and methods.

[0002] There is a technology that allows an external body, such as an avatar robot, to possess some of the functions of the human body, enabling interaction between the body and the environment. One such technology is telexistence technology, which allows an operator to control the avatar robot as if it were their own body, enabling physical activity in a remote location.

[0003] Since humans are born with only one body, it has been assumed that one operator will also operate one external body. In recent years, attempts have been made to have one operator operate multiple bodies. If the movements of multiple bodies can be controlled, it will be possible to simultaneously perform tasks that one wishes to perform with one's own physicality at multiple locations, and it is expected that the productivity of human physical activity will improve.

[0004] As a technique for controlling the motions of multiple bodies, there is a method for simultaneously controlling the motions of multiple bodies by synchronizing the motion of an operator with two other motions of the body, as disclosed in, for example, Non-Patent Document 1. This method suggests the possibility that a user may be able to control multiple bodies with a sense of ownership.

[0005] Ryota Kondo and Maki Sugimoto. "Investigating the simultaneity of the sense of ownership of multiple bodies in VR space using affordance tasks." Transactions of the Virtual Reality Society of Japan, Vol. 27, No. 4, 2022, pp. 353-360. <https: / / doi.org / 10.18974 / tvrsj.27.4_353>

[0006] Conventional methods only support tasks with perfectly matched trajectories between multiple bodies. However, conventional methods have the limitation that they cannot simultaneously input different trajectories of multiple body movements.

[0007] Therefore, in order to utilize multiple bodies for a variety of tasks, it is important to resolve these constraints and accommodate multiple tasks with different trajectories.

[0008] This invention was made in light of the above circumstances, and its purpose is to provide a motion control device and method that can control the trajectories of separate movements of multiple bodies.

[0009] A motion control device according to one aspect of the present invention includes a division unit that acquires information indicating the trajectory of a person's body movement along a third surface located between a first surface and a second surface that is a surface at an angle different from that of the first surface, and divides the trajectory indicated by the acquired information into a trajectory of movement along the first surface and a trajectory of movement along the second surface and outputs the trajectory; and a control unit that, when control of the movement of a first object that imitates the body movement is enabled, controls the movement of the first object along the trajectory of movement along the first surface output by the division unit, and, when control of the movement of a second object that imitates the body movement is enabled, controls the movement of the second object along the trajectory of movement along the second surface output by the division unit.

[0010] A motion control method according to one aspect of the present invention is a method performed by a motion control device, and includes: acquiring, by a division unit of the motion control device, information indicating a trajectory of a person's body movement along a third plane located between a first plane and a second plane that is a plane at an angle different from that of the first plane; dividing the trajectory indicated by the acquired information into a trajectory of movement along the first plane and a trajectory of movement along the second plane and outputting the divided trajectory; and controlling, by a control unit of the motion control device, when control of the movement of a first object that imitates the body movement is enabled, controlling the movement of the first object along the trajectory of movement along the first plane output by the division unit; and when control of the movement of a second object that imitates the body movement is enabled, controlling the movement of the second object along the trajectory of movement along the second plane output by the division unit.

[0011] According to the present invention, it is possible to control the trajectories of multiple separate body movements.

[0012] FIG. 1 is a diagram showing an application example of a motion control system according to an embodiment of the present invention. FIG. 2 is a diagram explaining an example of processing by an input trajectory data division unit. FIG. 3 is a diagram explaining the concept of calculating a speed ratio. FIG. 4A is a diagram showing an example of an output trajectory pattern. FIG. 4B is a diagram showing an example of an output trajectory pattern. FIG. 4C is a diagram showing an example of an output trajectory pattern. FIG. 4D is a diagram showing an example of an output trajectory pattern. FIG. 4E is a diagram showing an example of an output trajectory pattern. FIG. 5 is a diagram showing an example of the appearance of an input device. FIG. 6 is a flowchart showing an example of the processing operation procedure of a physical simultaneous control device according to an embodiment of the present invention. FIG. 7 is a block diagram showing an example of the hardware configuration of a physical simultaneous control device according to an embodiment of the present invention.

[0013] An embodiment of the present invention will now be described. FIG. 1 is a diagram showing an application example of a motion control system according to an embodiment of the present invention. In the example shown in FIG. 1, the motion control system according to an embodiment of the present invention includes a body simultaneous control device 100, which is a motion control device, and an input device 110. The body simultaneous control device 100 includes a calibration acquisition unit 10, a calibration storage unit 20, an input trajectory data division unit 30, a displacement calculation unit 40, an output trajectory data generation unit 50, and a trajectory data transmission unit 60. The calibration storage unit 20 has a storage device such as a non-volatile memory. The input device 110 may be included in the body simultaneous control device 100.

[0014] The physical simultaneous control device 100 of this embodiment controls the trajectories of multiple bodies moving on separate trajectories on different planes by operation inputs on a composite plane of a first plane and a second plane at an angle different from that of the first plane, in this case a third plane located at an angle between the first and second planes. The above-mentioned moving on separate trajectories on different planes can include moving at separate speeds on different planes. The second and third planes are planes obtained by fixing one side of the first plane and shifting the angle of this plane. This embodiment solves the problems of the prior art as follows (B-1) and (B-2).

[0015] (B-1) Simultaneous input of different trajectories of multiple bodies: Two trajectories of multiple bodies are operated together with a single input on a composite plane trajectory, and by switching the trajectory control ON / OFF, two trajectories with different starting and ending points are input.

[0016] (B-2) Simultaneous input of different speeds of multiple bodies: The speed ratio of each trajectory is controlled by the angle of the composite plane.

[0017] 1, for example, various robots, the work of which is targeted in this embodiment is to move a robot arm at a certain speed. The input device 110 can be, for example, a device such as a VR (Virtual Reality) controller that can acquire an input trajectory when the body is moved as three-dimensional coordinates and has buttons that can be used for input operations independently of the trajectory.

[0018] 1 are virtual avatars that are different from innate bodies and move in response to arbitrary trajectory data, but have the same degrees of freedom of joints as innate bodies. The bodies a and b are bodies whose movements are to be controlled, and for example, can freely move the positions of their hands according to trajectory data.

[0019] The calibration acquisition unit 10 acquires the initial position of the input trajectory, which is the initial position of the input device 110, and the initial positions of the movable parts to be operated of body a and body b, as the initial positions of the input trajectory, i.e., the initial positions of body a and body b, and stores these acquired initial positions in the calibration memory unit 20.

[0020] The input trajectory data dividing unit 30 divides the input trajectory data received from the input device 110 into trajectory data α of the component of trajectory control surface A of body a and trajectory data β of the component of trajectory control surface B of body b.

[0021] 2 is a diagram illustrating an example of processing by the input trajectory data dividing unit. In the example shown in FIG. 2, the trajectory data α of the component of the trajectory control surface A is ａThe trajectory data β of the component of the trajectory control surface B corresponds to the trajectory of the arm of the second robot r ｂ corresponds to the arm trajectory.

[0022] The displacement calculation unit 40 compares each piece of input trajectory data divided by the input trajectory data division unit 30 one-to-one with the initial position of the input trajectory stored in the calibration storage unit 20, and calculates the amount of displacement of the input trajectory data from the initial position of the input trajectory for each piece of trajectory data α and β. The amount of displacement calculated here is the difference between the trajectory data α and the initial position of the input trajectory (sometimes referred to as the amount of displacement related to the trajectory data α), and the difference between the trajectory data β and the initial position of the input trajectory (sometimes referred to as the amount of displacement related to the trajectory data β).

[0023] The output trajectory data generation unit 50 generates output trajectory data for body a by adding a displacement amount based on the difference between the trajectory data α and the initial position of the input trajectory, calculated by the displacement calculation unit 40, to a displacement amount that reflects the speed ratio of the trajectory and the ON / OFF of trajectory tracking for body a, to the initial position of the movable part to be operated for body a, stored in the calibration memory unit 20.

[0024] Furthermore, the output trajectory data generation unit 50 generates output trajectory data for body b by adding the displacement amount, which is calculated by the displacement calculation unit 40 and is based on the difference between the trajectory data β and the initial position of the input trajectory, and which reflects the speed ratio of the trajectory and the ON / OFF of trajectory tracking for body b, to the initial position of the movable part to be operated for body b stored in the calibration storage unit 20. The above-mentioned trajectory tracking refers to input to the output trajectory data generation unit 50.

[0025] The method for calculating the velocity ratio of the above orbit will now be described. Figure 3 is a diagram for explaining the concept of calculating the velocity ratio. As shown in Figure 3, the angle between orbit control plane A and orbit control plane B is θ ＡＢ The inclination of the composite plane from the orbit control surface A is θ (0≦θ≦θ ＡＢ ) and by changing the inclination θ of the composite plane, the velocity ratio between body a and body b is changed. ＡＢ -θ) is the velocity ratio between body a and body b.

[0026] Next, a method for switching ON / OFF of the trajectory tracking will be described. In this embodiment, by corresponding the pressing (ON) and release (OFF) of a specific button on the input device 110 to ON and OFF of the trajectory tracking, it is possible to control trajectories in which the start and end timings of the movements of body a and body b are different from each other.

[0027] 4A, 4B, 4C, 4D, and 4E are diagrams showing examples of output trajectory patterns. Here, examples are given of a trajectory pattern in which the start and end points of the trajectories of task T1 on orbit control surface A and task T2 on orbit control surface B are the same, as shown in Fig. 4A, and patterns in which at least one of the start and end points of the trajectories of task T1 on orbit control surface A and task T2 on orbit control surface B is different, as shown in Figs. 4B to 4E.

[0028] The trajectory pattern P1 shown in FIG. 4A, the trajectory pattern P2 shown in FIG. 4B, the trajectory pattern PC shown in FIG. 4C, the trajectory pattern P4 shown in FIG. 4D, and the trajectory pattern P5 shown in FIG. 4E are the trajectory patterns shown below.

[0029] P1: Full synchronization P2: Start point deviation (synchronized) P3: End point deviation (synchronized) P4: Start point and end point deviation (synchronized) P5: Start point and end point deviation (asynchronous)

[0030] Trajectory pattern P1 shown in Figure 4A is a pattern in which the start and end points of the trajectory of task T1 are fixed, and the start and end points of the trajectory are the same for tasks T1 and T2. Trajectory patterns P2 to P4 shown in Figures 4B to 4D are patterns in which the start and end points of the trajectory of task T1 are fixed, and at least one of the start and end points of the trajectory is different for tasks T1 and T2. Trajectory pattern P2 is a pattern in which task T2 starts later than task T1, and tasks T1 and T2 finish at the same time. Trajectory pattern P3 is a pattern in which tasks T1 and T2 start at the same time, and task T2 finishes before task T1. Trajectory pattern P4 is a pattern in which task T2 starts later than task T1, and task T2 finishes before task T1. Furthermore, trajectory pattern P5 shown in FIG. 4E is a pattern in which task T1 starts before task T2 and task T2 ends after task T1, i.e., unlike trajectory pattern P4, this pattern has sections in which the two trajectories are turned ON / OFF.

[0031] For the trajectory patterns P2 to P4, the output trajectory data generating unit 50 generates output trajectory data that corresponds the pressing (ON) and release (OFF) of a specific button on the input device 110 to the ON and OFF of trajectory tracking of the task T2, thereby controlling the corresponding trajectory patterns P2 to P4.

[0032] In other words, by mapping the start and end points of the trajectory of task T2 to the pressing and releasing of specific buttons on the controller, the operator can control the above trajectory patterns P2 to P4 by drawing a composite plane using the controller, which is the input device 110, while also operating the buttons on and off.

[0033] Fig. 5 is a diagram showing an example of the external appearance of the input device. Furthermore, when the start and end points of the trajectories are different between tasks T1 and T2 as shown in Fig. 4E, two buttons 110a and 110b are mounted on the input device 110, and output trajectory data generating unit 50 generates output trajectory data in which the depression (ON) and release (OFF) of these buttons correspond to the ON and OFF of trajectory tracking for both tasks T1 and T2, respectively, to control trajectory pattern P5, as shown in Fig. 5(a) and (b).

[0034] That is, the start and end points of the trajectory of task T1 are mapped to the pressing and releasing of a first button on the controller, for example, button 110a of the input device 110, and the start and end points of the trajectory of task T2 are mapped to the pressing and releasing of a second button on the controller, for example, button 110b of the input device 110. In this way, the operator can control the trajectory pattern P5 by turning on and off various buttons while drawing a composite plane using the controller, which is the input device 110.

[0035] The input device 110 may be a sensor attached to the wrist of the operator, such as a sensor-based motion capture using a VR tracker, or an optical motion capture using a depth camera.

[0036] In the sensor-based motion capture, the amount of rotation of the operator's wrist, which is not used to control the body, may be used to perform trajectory tracking control by turning on / off a button on the input device 110. In the optical motion capture, the trajectory tracking control may be performed using hand gestures of the operator's fingers, such as a fist and an open palm.

[0037] In addition to the virtual avatars described above, the above bodies a and b may be physical robots with the same degrees of freedom of joints as humans, or avatars or robots with degrees of freedom of joints different from humans.

[0038] When the degrees of freedom of joints differ from those of a human as described above, trajectory tracking control may be performed using data calculated by inverse kinematics based on input trajectory data for the relevant joint coordinates.

[0039] Next, an example of the procedure of the processing operation of the body simultaneous control device according to one embodiment of the present invention will be described. Figure 6 is a flowchart showing an example of the procedure of the processing operation of the body simultaneous control device according to one embodiment of the present invention. Here, it is assumed that the input device 110 is provided with the first and second buttons, and the ON / OFF of the operation trigger of the body a, i.e., the ON / OFF of trajectory tracking, corresponding to the ON / OFF of the first button, is reflected in the displacement amount of the trajectory data α, and the ON / OFF of the operation trigger of the body b, corresponding to the ON / OFF of the second button, is reflected in the displacement amount of the trajectory data β.

[0040] First, the calibration acquisition unit 10 acquires the initial position of the trajectory of the input device 110 and stores it in the calibration storage unit 20 (S11). Next, the input trajectory data division unit 30 receives input trajectory data of the composite plane from the input device 110 (S12), divides this input trajectory data into input trajectory data α of the component of trajectory control surface A and input trajectory data β of the component of trajectory control surface B, and outputs them to the displacement calculation unit 40 (S13).

[0041] The displacement calculation unit 40 receives the input trajectory data α and β from the input trajectory data division unit 30, and compares these input trajectory data with the initial positions of the trajectories stored in the calibration storage unit 20 to calculate the amount of displacement related to the trajectory data α and the amount of displacement related to the trajectory data β, and outputs the calculated amounts to the output trajectory data generation unit 50.

[0042] The output trajectory data generation unit 50 receives the displacement amount from the displacement calculation unit 40. The output trajectory data generation unit 50 generates output trajectory data for body a by adding, to the initial position of the input device 110, the displacement amount related to the trajectory data α from the displacement calculation unit 40, the displacement amount reflecting the ON / OFF of the operation trigger for body a, which corresponds to the ON / OFF of the first button of the input device 110, and generates output trajectory data for body b by adding, to the initial position of the input device 110, the displacement amount related to the trajectory data β from the displacement calculation unit 40, the displacement amount reflecting the ON / OFF of the operation trigger for body b, which corresponds to the ON / OFF of the second button of the input device 110 (S14).

[0043] In addition, a trigger ON / OFF detection unit may be provided in the body simultaneous control device 100 separate from the output trajectory data generation unit 50, and this detection unit may detect the ON / OFF of the operation trigger and reflect this in the displacement amount related to each of the above-mentioned trajectory data.

[0044] When the operation trigger for body a is ON (Yes in S15), the output trajectory data generation unit 50 reflects the operation of trajectory control surface A on body a (S16). S16 means that the output trajectory data generated for trajectory control surface A is transmitted from the trajectory data transmission unit 60 to body a.

[0045] When the answer is "No" in S15 or after S16, if the operation trigger for body b is ON (Yes in S17), the output trajectory data generation unit 50 reflects the operation of trajectory control surface B on body b (S18). S18 means that the output trajectory data generated for trajectory control surface B is transmitted from the trajectory data transmission unit 60 to body b.

[0046] If the answer is "No" in S17, or if the input device 110 is being operated on the composite plane after S18 (Yes in S19), the process returns to S12. On the other hand, if the answer is "No" in S19, the series of processes ends.

[0047] In the embodiment described above, by controlling the trajectories of multiple bodies moving on separate trajectories on different planes with operation input on a composite plane, it is possible to simultaneously acquire the different trajectories of multiple bodies as input trajectory data. Then, by switching the buttons of the input device ON and OFF corresponding to the ON and OFF of the body operation trigger, it is possible to control multiple body movements with different movement timings.

[0048] 7 is a block diagram showing an example of the hardware configuration of a motion control device according to an embodiment of the present invention. In the example shown in FIG. 7, the physical simultaneous control device 100 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to this hardware processor 111A via a bus 115.

[0049] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0050] An input device 200 and an output device 300 attached to the physical simultaneous control device 100 and used by a user or the like are connected to the input / output interface 113. The input device 200 corresponds to the input device 110 described above. The input / output interface 113 can acquire operation data input by a user or the like through the input device 200, such as a keyboard, touch panel, touchpad, or mouse, and can output and display the output data to an output device 300, such as a display device using liquid crystal or organic electroluminescence (EL) display. The input device 200 and the output device 300 may be devices built into the physical simultaneous control device 100, or may be input devices and output devices of other information terminals that can communicate with the physical simultaneous control device 100 via a network.

[0051] The program memory 111B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and can store programs necessary to execute various control processes, etc., according to one embodiment.

[0052] The data memory 112 is a tangible storage medium that is, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and can be used to store various data or information acquired and created during various processing steps.

[0053] The physical simultaneous control device 100 according to one embodiment of the present invention can be configured as a data processing device having the respective units shown in FIG. 1 as software processing functional units.

[0054] The information storage unit used as a work memory or the like by each part of the physical simultaneous control device 100 can be configured using the data memory 112 shown in Fig. 7. However, these configured storage areas are not essential components within the physical simultaneous control device 100, and may be areas provided in, for example, an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0055] The processing function units in each of the above units can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0056] The methods described in each embodiment can be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), or semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes storage media such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0057] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0058] REFERENCE SIGNS LIST 100: Body simultaneous control device 110: Input device 10: Calibration acquisition unit 20: Calibration storage unit 30: Input trajectory data division unit 40: Displacement calculation unit 50: Output trajectory data generation unit 60: Trajectory data transmission unit

Claims

1. A motion control device comprising: a division unit that acquires information indicating a trajectory of a person's body movement along a third plane located between a first plane and a second plane that is a plane at an angle different from that of the first plane, and divides the trajectory indicated by the acquired information into a trajectory of movement along the first plane and a trajectory of movement along the second plane, and outputs the divided trajectories; and a control unit that, when control of a first object's movement that imitates the body movement is enabled, controls the first object's movement along the trajectory of movement along the first plane output by the division unit, and, when control of a second object's movement that imitates the body movement is enabled, controls the second object's movement along the trajectory of movement along the second plane output by the division unit.

2. The motion control device according to claim 1, wherein the control unit controls the motion of the first object and the motion of the second object using a speed ratio based on the inclination of the third surface relative to one of the first and second surfaces.

3. A motion control device as described in claim 1, further comprising a displacement calculation unit that calculates a displacement amount of the trajectory of the motion along the first surface by comparing a current position of the trajectory of the motion along the first surface output by the division unit with an initial position of the acquired trajectory, and calculates a displacement amount of the trajectory of the motion along the second surface by comparing a current position of the trajectory of the motion along the second surface output by the division unit with an initial position of the acquired trajectory, wherein the control unit, when control of the motion of a first object that imitates the motion of the body is enabled, controls the motion of the first object along the trajectory of the motion along the first surface obtained by adding the displacement amount of the trajectory of the motion along the first surface calculated by the displacement calculation unit to the initial position of the first object, and when control of the motion of a second object that imitates the motion of the body is enabled, controls the motion of the second object along the trajectory of the motion along the second surface obtained by adding the displacement amount of the trajectory of the motion along the second surface calculated by the displacement calculation unit to the initial position of the second object.

4. A method performed by a motion control device, comprising: acquiring, by a division unit of the motion control device, information indicating a trajectory of a person's body movement along a third plane located between a first plane and a second plane that is at an angle different from that of the first plane; dividing the trajectory indicated by the acquired information into a trajectory of movement along the first plane and a trajectory of movement along the second plane, and outputting the divided trajectories; and controlling, by a control unit of the motion control device, when control of the movement of a first object that imitates the body movement is enabled, the movement of the first object along the trajectory of movement along the first plane output by the division unit, and when control of the movement of a second object that imitates the body movement is enabled, the movement of the second object along the trajectory of movement along the second plane output by the division unit.

Images