Haptic Feedback System for Embodied Swarm Robots

Abstract

To make it easier to embody swarm robots, we provide a swarm robot control system equipped with a system that provides tactile feedback to the operator. [Solution] Target position coordinates are set for multiple robots that can move freely within a set field. Each robot moves toward its target position coordinates. Feature points on the operator's body are detected, and a value equivalent to the amount of movement of the feature point's position coordinate is assigned to the corresponding robot's target position coordinates. This causes each robot to behave in a way that follows the corresponding feature point. When the robots come into contact or collide, this is detected, and modules attached to the feature points corresponding to the robots output force, vibration, movement, or a combination of these, providing tactile feedback to the operator.

Description

[Technical Field]

[0001] The present invention relates to a control system for swarm robots with tactile feedback. [Background technology]

[0002] The human brain's plasticity and flexibility allow it to recognize objects as parts of the body and, under certain conditions, to intuitively manipulate them. This phenomenon is called embodiment. Utilizing this cognitive ability, various embodied systems have been developed that offer intuitive operation and an immersive feeling, such as remote control of virtual avatars and robotic arms. The sense of embodiment is a subjective sense of control, of being aware that oneself is the one causing a certain movement or event, and is composed of a "sense of agency," which is consciousness, and a "sense of ownership," which is the feeling of connection between oneself and an object.

[0003] Swarm robots are robots that coordinate the actions of multiple robots for a specific purpose. Due to their flexibility, scalability, and robustness, swarm robots are expected to be used in a wide variety of applications, from business to entertainment, and are a field that has seen continuous research and development in recent years. We focused on the combination of embodied robots and swarm robots.

[0004] Experiments on operating swarm robots have already proven that humans can embody them. However, due to individual differences in sensory perception and delays caused by collisions between robots, a swarm robot control system that can be embodied by many people has yet to reach a practical level. Therefore, we propose a system in which a swarm robot controlled by the movement of a feature point on the operator's body provides tactile feedback to the operator by outputting force, vibration, movement, or a combination of these to the feature point when the robot comes into contact with or collides with the operator. Here, tactile feedback refers to feedback from the robot to the operator that stimulates the operator's sense of touch. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-194948

[0006] [Patent Document 2] Japanese Patent Application Publication No. 2019-209425 Summary of the Invention [Problem to be solved by the invention]

[0007] The problem we are trying to solve is that swarm robots that many people can embody have not been realized due to delays caused by collisions between swarm robots and individual differences in sensory perception. Specifically, the only way to understand the situation of a swarm robot is through visual information. [Means for solving the problem]

[0008] The present invention provides a swarm robot control system that includes a plurality of robots that can move freely within a set field, a module that detects feature points on the body of a driver and acquires the position coordinates, a module that controls the plurality of robots based on changes in the position coordinates of the feature points, a module that acquires the position coordinates of each of the plurality of robots, a module that detects contact or collision between each robot and an object within the field or between each robot, and a module that provides tactile feedback by outputting force, vibration, movement, or a combination thereof to the feature points on the body of the driver that correspond to the robot that made contact or collided when the module detects contact or collision.The swarm robot control system is characterized in that, in a swarm robot controlled by changes in the position coordinates of the feature points on the body of the driver, when contact or collision between the robots is detected, force, vibration, movement, or a combination thereof is output to the feature points corresponding to the robot that made contact or collided, thereby providing tactile feedback to the driver. [Effects of the Invention]

[0009] The swarm robot operation system of the present invention using tactile feedback has the advantage of improving the sense of embodiment and making it easier to embody by providing tactile feedback to the operator by outputting force, vibration, or movement, or a combination thereof, in accordance with the status of the swarm robot, which is controlled by changes in the position coordinates of the body's feature points. In this invention, a module that detects contact or collision between each robot sends a signal when it detects contact or collision between each robot, which is transmitted via a computer to a module that provides tactile feedback to the operator, worn on the operator's feature points, and the module outputs force, vibration, or movement, or a combination thereof, allowing the operator to obtain tactile feedback, making it possible to grasp the status of the swarm robot based on tactile information even in the absence of visual information. [Brief explanation of the drawings]

[0010] [Figure 1] A diagram showing the modules required to control the swarm robots and the swarm robots initially positioned at random locations. [Figure 2] A diagram showing the flow of information required to set the target position coordinates for rearranging the swarm robots into a hand shape. [Figure 3] A diagram showing the robot's movement methods and possible movement and rotation directions. [Figure 4] A diagram showing the target position coordinates set within the field for the swarm robots to line up in the shape of a hand. [Figure 5] A diagram showing swarm robots arranged in the shape of a hand [Figure 6] The diagram shows how each feature point moves and the corresponding target position coordinates move, and how each robot moves to each target position coordinate. [Figure 7] A diagram showing the amount of movement of the x-axis and y-axis coordinates of P-1 due to the movement of P-1 [Figure 8] A diagram showing the amount of movement of the x-axis and y-axis coordinates of T-1 in accordance with the movement of P-1. [Figure 9]A flowchart showing the feedback control until R-1 reaches T-1. [Figure 10] A diagram showing a top view of the robot and the pressure sensors installed on the robot. [Figure 11] This diagram shows two robots colliding with an object in the field, transmitting this information wirelessly to the control PC, which then controls the vibration motors attached to the two feature points corresponding to the two colliding robots. [Figure 12] A diagram showing the positions of the operator, object A, and swarm robots in Field 2 [Explanation of symbols]

[0011] 1. Control PC 2.Transmitting information from C-1 to the control PC 3.Transmitting information from C-2 to the control PC C-1. Camera that detects the feature points of the left hand and transmits them to the control PC C-2. Camera to acquire the position coordinates of each robot F-1.C-1 detection range Area where F-1 and C-2 can be detected and where the swarm robots can move P-1. Detected feature points of the little finger P-2. Detected feature points of the ring finger P-3. Detected middle finger feature points P-4. Detected index finger feature points P-5. Detected thumb feature points R-1. One of the swarm robots that corresponds to P-1 R-2. One of the swarm robots that corresponds to P-2 R-3. One of the swarm robots that corresponds to P-3 R-4. One of the swarm robots that corresponds to P-4 R-5. One of the swarm robots that corresponds to P-5 4. Bottom view of one robot 5. Front view of one robot 6. Two wheels mounted on the robot 7.6 Rotation Direction 8. Possible movement and rotation directions of the robot T-1.R-1 target position coordinates T-2.R-2 target position coordinates T-3.R-3 target position coordinates T-4.R-4 target position coordinates T-5.R-5 target position coordinates W-1. Control signals are transmitted wirelessly from the control PC to each robot. AP-1. Coordinates of P-1 after movement AP-2. Coordinates of P-2 after movement AP-3. Coordinates of P-3 after moving AP-4. P-4 coordinates after movement AP-5. Coordinates of P-5 after moving BP-1. Coordinates of P-1 before movement BP-2. Coordinates of P-2 before movement BP-3. Coordinates of P-3 before moving BP-4. Coordinates of P-4 before moving BP-5. Coordinates of P-5 before moving dx-1. Any difference between the x-axis coordinates of AP-1 and BP-1, any movement amount of the x-axis coordinate of P-1 dy-1. Any difference in the y-axis of AP-1 and BP-1, any movement amount of the y-axis coordinate of P-1 AT-1. Coordinates of T-1 after movement BT-1. Coordinates of T-1 before movement Fx-1. The size of the x-axis of F-1 Fx-2. The size of the x-axis of F-2 Fx-2 / Fx-1. The x-axis ratio of F-1 to F-2 expressed as a fraction dx-1×(Fx-2 / Fx-1). The amount of movement of the x-axis of the target position coordinate of T-1 when the arbitrary amount of movement of the x-axis coordinate of P-1 is dx-1 Fy-1. The size of the y-axis of F-1 Fy-2. The size of the y-axis of F-2 Fy-2 / Fy-1. The y-axis ratio of F-1 to F-2 expressed as a fraction. dy-1×(Fy-2 / Fy-1). The amount of movement of the y-axis of the target position coordinate of T-1 when the arbitrary amount of movement of the y-axis coordinate of P-1 is dy-1 9. Top view of one robot 10. Four pressure sensors installed on one robot M-1. Vibration motor attached to the little finger M-2. Vibration motor attached to the ring finger M-3. Vibration motor attached to the middle finger M-4. Vibration motor attached to the index finger M-5. Vibration motor attached to the thumb Objects in OF-2 Collision between SO and R-1,2 W-2. Information is transmitted wirelessly from the pressure sensor that detected the collision to the control PC. 11. Control signal from control PC to M-1, 2 12. Pilot R. Swarm Robots A. Objects in F-2 that can be moved by swarm robots DETAILED DESCRIPTION OF THE INVENTION

[0012] Due to individual differences in embodied sensations and contact or collision between swarm robots, it has not yet been possible to put swarm robots into practical use that anyone can embody.To solve this problem, we invented a swarm robot control system that includes a swarm robot controlled by changes in the position coordinates of characteristic points on the body and a module that provides tactile feedback to the operator.By providing tactile feedback to the operator, rather than simply controlling the swarm robot unilaterally from the operator, mutual interaction occurs between the operator and the swarm robot, enabling more intuitive and immersive operation, in other words, improving the sense of embodiment.

[0013] Detect any feature points on the bodies of one or more operators and acquire their coordinates using a module that controls the swarm robots. Feature points here refer to fingertips, joints, or any point on the body. A PC is used to create a one-to-one correspondence between each feature point and each swarm robot. At this time, the number of feature points detected must match the number of swarm robots. The number can be either the number of swarm robots being handled or the number of detectable body feature points. Furthermore, a system that acquires the absolute position coordinates of any swarm robot can acquire the absolute position coordinates of each swarm robot, and the field is the range within which the swarm robot can move using any movement method. The coordinates at which each robot is placed are defined as the initial absolute position coordinates and initial target position coordinates of each swarm robot.

[0014] The control PC is connected to a module that detects any body feature point and controls the swarm robot, and a module that acquires the absolute position coordinates of any of the swarm robots. Once the swarm robots are deployed, they can be operated. The system records the change in any position coordinate and the amount of movement, and assigns an equal amount of movement to the target position coordinates of each swarm robot corresponding to each feature point, moving the target position coordinates. The swarm robot then moves in a straight line to the target position coordinates using any movement method. At this time, a system that acquires the absolute coordinates of any of the swarm robots calculates the difference between each swarm robot's position and each target position coordinate, confirms whether each target position coordinate has been reached, and performs feedback control. When each swarm robot reaches its target position coordinate, it stops in place and waits until the target position coordinate moves again. As each robot moves toward the target position coordinates corresponding to each feature point, it behaves as if it is following the corresponding feature point.

[0015] Each robot is equipped with a module that provides tactile feedback by outputting any force, vibration, or movement, or a combination of these, to each feature point. When the swarm robots come into contact with or collide with each other, or with an object in the field, a module that detects contact or collision between any swarm robot and the object detects it and transmits it to the control PC using any method. The control PC then activates the module that provides tactile feedback for each feature point corresponding to the robot that came into contact or collided, and provides tactile feedback to the operator's corresponding feature point. The robot that came into contact or collided continues to move toward its target position coordinates. Furthermore, even after providing feedback, the operator continues to control the swarm robot until it has completed its purpose using the robot. [Example]

[0016] An embodiment will be described using the drawings. 1 in Figure 1 is a control PC. C-1 in Figure 1 is a module for detecting feature points on the operator's body, such as a camera (hereinafter referred to as camera 1), and 2 in Figure 1 shows the flow of wired information transmission from camera 1 to the control PC. P-1, 2, 3, 4, and 5 in Figure 1 are, for example, five feature points detected by camera 1 on the fingertips of a left hand. P-1 corresponds to the tip of the little finger, P-2 the tip of the ring finger, P-3 the tip of the middle finger, P-4 the tip of the index finger, and P-5 the tip of the thumb. These feature points will be referred to as P-1, 2, 3, 4, and 5, respectively, hereinafter. F-1 in Figure 1 is the detection range of camera 1 (hereinafter referred to as field 1). When P-1, 2, 3, 4, and 5 are within field 1, their position coordinates are acquired and transmitted to the control PC as shown in 2 in Figure 1. In Figure 1, R-1, 2, 3, 4, and 5 are the five robots to be handled. Hereafter, the five robots as a whole will be referred to as the swarm robot, and each robot will be referred to as R-1, 2, 3, 4, and 5, respectively. R-1 corresponds to P-1, R-2 to P-2, R-3 to P-3, R-4 to P-4, and R-5 to P-5. The swarm robot is initially placed in an arbitrary location. C-2 in Figure 1 is a camera (hereafter referred to as camera 2) used to acquire the position coordinates of the swarm robot. Camera 2 is installed facing vertically downward so that it can observe the swarm robot from above. F-2 in Figure 1 is the area that can be detected by camera 2 and in which the swarm robot can move. Hereafter, this will be referred to as field 2. 3 in Figure 1 shows the flow of information transmitted via wire from camera 2 to the control PC.

[0017] Figure 2 explains the robots we will be dealing with. The movement mechanism of each robot is, for example, a two-wheeled system that moves forward, backward, rotates, and travels on a flat surface. Figures 4 and 5 in Figure 2 show the back and front views of the robot, respectively. Figure 2 shows the wheels of the robot, and Figure 2 shows the direction of rotation of the wheels. Figure 2 shows the direction of movement by forward and backward, and the direction of rotation, and the direction of movement that combines forward, backward, and rotation. In this way, each robot can rotate, move forward, backward, and move in a combination of these ways, depending on the movement of its wheels.

[0018] As shown in Figure 3, camera 1 transmits the position coordinates of each feature point and the size of Field 1 (xy plane), and camera 2 transmits the size of Field 2 (xy plane) to the control PC. Then, the ratio of the size of Field 1 from Camera 1 to the size of Field 2 is calculated, and the target position coordinates are set on Field 2 so that they form a hand-shaped arrangement based on the position coordinates of each feature point in Field 1 and their ratio. In Figure 4, T-1, 2, 3, 4, and 5 are the target position coordinates of R-1, 2, 3, 4, and 5. Hereafter, each target position coordinate will be referred to as T-1, 2, 3, 4, and 5. Furthermore, information is transmitted wirelessly from the control PC to the swarm robots, and the target position coordinates of each robot are set. In Figure 4, W-1 represents the wireless transmission of control signals from the control PC to each robot. After the target position coordinates are set, each swarm robot moves straight toward its target position coordinate. Figure 5 shows the swarm robots arriving at their target position coordinates and forming a hand-shaped arrangement. When each robot arrives at its set target coordinates and the swarm robots form a hand-shaped arrangement, the operator can control the swarm robots. By forming the swarm robots in the same hand-shaped arrangement as the operator, the swarm robots can move more like humans, enabling intuitive and immersive operation.

[0019] The operator controls the swarm robot by changing the position coordinates of the feature points, i.e., by moving the feature points. When the operator is able to control the swarm robot, each target position coordinate is the same as the position coordinate of each robot. In Figure 6, AP-1, 2, 3, 4, and 5 are the position coordinates of P-1, 2, 3, 4, and 5 after movement, respectively, and BP-1, 2, 3, 4, and 5 are the position coordinates of P-1, 2, 3, 4, and 5 before movement, respectively. In Figure 7, dx-1 is the difference between any x-coordinates of BP-1 and AP-1, that is, the amount of movement of any x-coordinate of P-1. Similarly, dy-1 in Figure 7 is the difference between any y-coordinates of BP-1 and AP-1, that is, the amount of movement of any y-coordinate of P-1. In Figure 8, BT-1 and AT-1 are the position coordinates of T-1 before and after movement, respectively. In Figure 8, Fx-1 and Fy-1 are the x-axis and y-axis sizes of the detection range of camera 1, respectively. Similarly, Fx-2 and Fy-2 in Figure 8 represent the x-axis and y-axis dimensions of the field size, respectively. Using Fx-1, Fy-1, Fx-2, and Fy-2, the x-axis and y-axis dimensions of the field size can be expressed as (Fx-2 / Fx-1) and (Fy-2 / Fy-1) times the x-axis and y-axis dimensions of camera 1's detection range, respectively. When the movement amounts of the feature point on the x-axis and y-axis are dx-1 and dy-1, respectively, the target position coordinates of R-1 corresponding to P-1 move by dx-1 × (Fx-2 / Fx-1) along the x-axis and dy-1 × (Fy-2 / Fy-1) along the y-axis. R-1 then moves straight toward the moved T-1. Similarly, the other robots R-2, 3, 4, and 5 are controlled by moving the corresponding feature points P-2, 3, 4, and 5 in any direction and any distance, measuring the amount of movement of each x-axis and y-axis coordinate, and then multiplying the amount of movement of each x-axis and y-axis coordinate by (Fx-2 / Fx-1) and (Fy-2 / Fy-1) to move the corresponding target position coordinates T-2, 3, 4, and 5 in the x-axis and y-axis directions, respectively, and moving each robot corresponding to each target position coordinate in a straight line. Figure 9 shows the flowchart of R-1 moving toward T-1 and reaching the target position coordinates. R-1 stops moving when it reaches T-1, stops there, and waits for T-1 to move again. Camera 2 is used to check whether the coordinates of R-1 and T-1 match. The same process is used for each of the other robots.The feature point is moved, and based on that amount of movement, the target position coordinates are moved, and the robot moves toward the target position coordinates. By repeating this process at shorter intervals, each robot can take action with greater freedom.

[0020] Each robot uses a pressure sensor, for example, as a module to detect contact or collision. 9 in Figure 10 shows a top view of one robot, and 10 in Figure 10 is a pressure sensor, mounted on each of the robot's four sides. The other four robots are similar. M-1, 2, 3, 4, and 5 in Figure 11 are vibration motors, for example, attached to the fingertips (hereafter referred to as M-1, 2, 3, 4, and 5, respectively). M-1 is attached to the little finger, M-2 to the ring finger, M-3 to the middle finger, M-4 to the index finger, and M-5 to the thumb. M-1, 2, 3, 4, and 5 are controlled by a control PC, for example, via a wired connection. O in Figure 11 is an object on the field (hereafter referred to as the object). S in Figure 11 indicates a collision between the object and R-1 and R-2. When the pressure sensors mounted on R-1 and R-2 detect a collision, the information is transmitted wirelessly to the control PC. W-2 in Figure 11 shows how information is transmitted wirelessly from the pressure sensor that detected the collision to the control PC. 11 in Figure 11 corresponds to R-1,2, and shows how a control signal is transmitted by wire from the control PC to M-1,2. After the collision between R-1,2 is detected, the information is transmitted to the control PC, which then activates the vibration motor, which outputs vibrations. This allows the operator to receive tactile feedback. Even after the collision, R-1,2 continues to travel toward T-1,2.

[0021] This swarm robot is controlled by this control system with the primary purpose of pushing and moving objects in Field 2. Whether the swarm robot can move an object is determined by its weight. A in Figure 12 is an object in Field 2 that can be moved by the swarm robot (hereafter referred to as Object A). R in Figure 12 is the swarm robot. 12 in Figure 12 is the operator. When the swarm robot and Object A come into contact or collide, tactile feedback is given to the operator. When the operator, Object A, and swarm robot are positioned as shown in Figure 12, Object A can be moved closer to the operator by moving the feature point closer to the operator so that the target position coordinates of the swarm robot are below the operator.

Claims

1. A plurality of robots that can move freely within a set field (called Field A), Module A detects the characteristic points of the pilot's body and acquires the position coordinates; a module C for acquiring the position coordinates of each of the plurality of robots; a module B that controls the plurality of robots based on the difference between target position coordinates of the plurality of robots that change in response to changes in the position coordinates of the feature points acquired by the module A and the position coordinates of each of the plurality of robots acquired by the module C; a module D for detecting contact or collision between each robot and an object in the field A or between each robot; a module E, worn on a feature point of the operator's body corresponding to each of the plurality of robots that has come into contact or collision with the module D, for providing tactile feedback by outputting force, vibration, movement, or a combination thereof to the feature point of the operator's body when the module D detects a collision or contact; A swarm robot control system with haptic feedback for embodied swarm robots, comprising:

2. The pilot's body feature points are located within a set field (called Field B), Module A is capable of acquiring position coordinates of the feature points within field B; Module B can move the corresponding robot to the target position coordinates in accordance with the movement of the feature point. The swarm robot control system with haptic feedback for the embodiment of the swarm robot according to claim 1.

3. The control system for swarm robots with haptic feedback for embodiment of swarm robots according to claim 2, characterized in that the position coordinates of the feature points in field B acquired by module A are sent to module B, and module B moves the corresponding robot to the target position coordinates in accordance with the movement of the feature points.

4. The swarm robot control system with haptic feedback for embodiment of swarm robots described in claim 2, characterized in that module A is a camera that is placed and is capable of acquiring position coordinates within field B of feature points on the body of an operator placed within field B.

5. The swarm robot control system with haptic feedback for embodied swarm robots according to claim 1, characterized in that module C is a camera that is placed and is capable of acquiring the position coordinates of each of multiple robots placed within field A.

6. The swarm robot control system with haptic feedback for swarm robot embodiment of claim 1, wherein module D is a pressure sensor.

7. The swarm robot control system with haptic feedback for embodiment of swarm robots as described in claim 1, characterized in that when module D detects a collision or contact, a signal that module D has detected the collision or contact is sent to module E, which then causes module E to apply force, vibration, or movement to a characteristic point on the operator's body.

8. The swarm robot control system with haptic feedback for swarm robot embodiment described in claim 1, characterized in that the body feature points are fingertips or finger joints.

Images