Wearable tactile feedback apparatus for man-machine formation control
By designing a wearable haptic feedback device, three forms of haptic feedback are used to provide real-time human-machine formation information, which solves the problem of insufficient visual feedback, improves the operator's perception ability and positioning efficiency, and reduces the difficulty of human-machine interaction.
Patent Information
- Application Number
- PCT/CN2024/079280
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-02-29
- Publication Date
- 2024-12-05
AI Technical Summary
In human-robot formation systems, due to the increased number of robots and the high load on the visual feedback channel, users may not receive effective visual feedback, making it difficult for operators to quickly locate task targets in complex environments.
A wearable haptic feedback device for human-machine formation control was designed. It adopts a small wireless wearable design and includes an interactive interface module, a main control module, a pressure feedback module, a shear force feedback module, and a vibration feedback module. It provides real-time formation, direction, and obstacle information of human-machine formation through three haptic feedback methods, making up for the shortcomings of visual feedback.
It enhances operators' awareness of the environment and formation status, provides directional guidance, helps users quickly locate mission objectives in complex environments, reduces the difficulty of human-computer interaction, and creates a highly immersive experience.
Smart Images

Figure CN2024079280_05122024_PF_FP_ABST
Abstract
Description
A wearable tactile feedback device for human-machine formation control Technical Field
[0001] The present invention belongs to the technical field of human-machine interaction, and in particular relates to a wearable tactile feedback device for human-machine formation control. Background Art
[0002] The research on human-robot formation systems has attracted increasing attention from scientific researchers in recent years. A human-robot formation system composed of multiple robots with relatively simple structures and functions can complete tasks that are difficult for a single robot to accomplish through complementary capabilities and coordinated actions among members, and has the advantages of good robustness and scalability.
[0003] Currently, visual feedback is a common way to convey human-robot formation information to users. However, in actual operation scenarios, due to the increase in the number of robots and the high load of the visual feedback channel and visual occlusion, users may not be able to obtain effective feedback in the visual channel, which requires further improvement.
[0004] Summary of the Invention
[0005] To solve the above problems, the present invention discloses a wearable tactile feedback device for human-machine formation control. It adopts a small wireless wearable design. Each module is worn on the arm of the user's auxiliary operator in a modular and split manner. It can generate three types of tactile forces on the user's skin surface. The tactile feedback information generated can serve as an effective supplement to the operator's visual and auditory perception, improving the operator's ability to perceive the surrounding environment and the current formation situation, and providing directional guidance to guide the operator to quickly locate the mission target in a complex environment.
[0006] To achieve the above object, the technical solution of the present invention is as follows:
[0007] A wearable tactile feedback device for human-machine formation control, comprising an interactive interface module and a main control module;
[0008] The interactive interface module includes an input module and a wearable tactile feedback module;
[0009] The wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module and a vibration feedback module, and these three modules are connected to the main control module;
[0010] The main control module uses a motherboard to connect the core board, Bluetooth module, and power supply, centralizing all inputs and outputs. This allows the main control module to control each of the three modules via connecting cables and obtain status information about the multi-robot system via the Bluetooth module. The mainboard utilizes L9110S and ULN2803 chips to drive four reduction motors and 25 vibration motors. The mainboard also includes a Bluetooth chip interface, enabling wireless control via Bluetooth serial communication. The connected power supply uses an 18650 lithium battery to provide a stable 7.4V voltage to the main control module and tactile feedback module.
[0011] The squeeze force feedback module consists of a servo and a strap. The retractable arm strap, used to generate squeeze tactile feedback, is connected to a rotating member. By controlling the rotation of the servo, the rotating member rotates, and the tightness of the strap is controlled, producing different squeeze forces on the user's forearm. The relationship between the servo's rotation angle and the applied pressure is as follows:
[0012] Where N is the squeezing force, θ is the servo rotation angle, k is the elastic modulus of the armband, R0 is the original radius of the armband, and R is the arm radius. When both servos rotate simultaneously to tighten the armband, the armband contracts, increasing the squeezing force on the user's forearm. This device is used to provide feedback on the expansion and contraction of the human-robot formation, providing real-time feedback on the formation's size based on the force of the armband's contraction.
[0013] The shear force feedback module consists of four DC reduction motors secured to the arm via printed components and straps. Friction wheels are mounted on the rotating shafts. The motors drive the friction wheels to generate shear force on the arm in four directions: forward, backward, left, and right. This provides feedback on the movement direction of the human-machine formation system.
[0014] The vibration feedback module consists of a vibration motor array. This array of 25 flat motors is fixed to a flexible circuit board. The user secures it to the back of their forearm and tightens it until they feel the motors contact their skin. This module provides feedback on formation changes and direction of movement during human-robot formation by providing different vibration patterns and specific orientations.
[0015] In order to achieve tactile feedback for human-machine formation control, the design principle of the present invention is:
[0016] 1. Design with three forms of tactile feedback
[0017] Haptic feedback offers unique advantages over other sensory channels. Human touch is not only the most robust and widespread sensory channel, but is also sensitive, bidirectional, and private. This wearable device generates three types of tactile forces on the user's skin surface, providing four types of feedback regarding the control of the human-robot formation system, addressing some of the practical limitations of visual feedback. This system effectively helps users perceive their surroundings and the human-robot formation, providing directional guidance to quickly locate their mission objectives.
[0018] Feedback 1: Contraction and expansion of the human-robot formation: This feedback is processed by the squeeze force module. The system's contraction scale corresponds to the rotation scale of the servos. When the user changes the size of the formation, the system immediately sends the corresponding servo angle to the squeeze module. Both servos work simultaneously to contract the straps, causing the user to feel the squeeze force generated by the contraction.
[0019] Feedback 2: Task Direction Guidance: This feedback is implemented through the shear force module. The shear module provides real-time feedback on the relative direction and distance between the parent / child target points and the operator, guiding the operator to the target points. Different relative directions between the operator and the parent / child targets cause the shear module to generate different shear forces. Different relative distances cause the shear module's reduction motor to rotate at different speeds, resulting in different shear forces.
[0020] Feedback information three includes information about the formation change of the human-machine team. Using the vibration array module, different shapes are drawn on the operator's arm using the tactile illusion of apparent motion. Apparent motion refers to the sensation of continuous vibration created by discrete vibrating points on the surface of human skin that vibrate at a specific duration and interval.
[0021] Feedback information 4: Obstacle direction information: Vibration motors around the vibration array module detect an obstacle approaching a critical distance. When a robot in a certain formation detects an obstacle, the surrounding vibration motors generate vibrations in that direction to alert the operator that the robot formation has encountered an obstacle. This helps the operator maintain a sense of the progress of the human-robot formation behind them while focusing on the front.
[0022] 2. Lightweight modular design
[0023] The main control unit is integrated into a single unit, while the modules are separately wearable on the user's secondary operator's arm (typically the left hand). The user can easily put it on and take it off, simply turning on the power to connect to the human-machine teaming system and begin operation. This split design helps reduce interference between different tactile feedback and improves the user's ability to distinguish between different feedback types. While wearing the device, the user can still use the wearing hand normally for various tasks, as tested in a simulation environment.
[0024] 3. Use self-designed circuits and realize communication
[0025] The system utilizes a proprietary circuit module connection mechanism, designed to connect the microcontroller core board and the individual modules. The circuit design includes driver chips for each module's motors, which directly drive them. A Bluetooth communication module also enables wireless serial communication with the microcontroller chip, controlling each module's response to specific situations. Due to its low power consumption, the system can be powered by a small lithium battery module, ensuring both compactness and long battery life.
[0026] The beneficial effects of the present invention are:
[0027] (1) The present invention adopts a real-time information feedback mechanism for human-machine formation based on three tactile feedback principles. The tactile feedback information generated by the mechanism can serve as an effective supplement to the operator's visual and auditory perception, thereby improving the operator's ability to perceive the surrounding environment and the current formation situation, and providing directional guidance to guide the operator to quickly locate the mission target in a complex environment.
[0028] (2) Tactile feedback has unique advantages over other sensory channels. Human touch is not only the most robust and widespread sensory channel, but also sensitive, bidirectional, and private. Therefore, compared with visual and auditory channels, tactile feedback has greater advantages in providing the wearer with feedback on the wearer's understanding of the external limbs and task status. It can play a unique role in reducing the difficulty of human-computer interaction and creating a highly immersive operating experience for users.
[0029] (3) The present invention adopts a small wireless wearable design. The main control device is integrated into one body, and each module is wearable on the arm of the operator and the assistant operator in a separate manner. The operator can put it on and take it off very conveniently. The operator only needs to turn on the power to connect to the human-machine formation system and start operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] FIG1 is a schematic diagram of the working of a test device system according to an embodiment of the present invention;
[0031] FIG2 is a hardware connection diagram of the device;
[0032] Figure 3 is an exploded structural diagram of the main control module;
[0033] FIG4 is a structural diagram of an extrusion force module and a shear force module;
[0034] Figure 5 is a structural diagram of the vibration array module.
[0035] List of accompanying figures: 1. Main control module frame; 2. Arduino core board; 3. Power supply interface of main control module; 4. Main control module mainboard; 5. Bluetooth chip interface; 6. Extrusion force feedback module and shear force feedback module expansion interface; 7. Vibration feedback module expansion interface; 8. 20Kg servo; 9. Extrusion force feedback module and shear force feedback module expansion board; 10. Gear motor; 11. Tape pulley; 12. Bearing; 13. Friction wheel for shear force feedback, 14. Shear force feedback module in the front and rear directions. DETAILED DESCRIPTION
[0036] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.
[0037] The present invention designs a wearable device with three forms of tactile feedback based on a multi-mobile robot collaborative control system, uses a VR controller as the command input interface for the collaborative control of the human-machine formation; and combines multiple force tactile information to provide feedback on the motion state of the human-machine formation.
[0038] As shown in Figure 1, it includes an interactive interface module, a communication module and a main control module. The interactive interface module includes a VR controller module for inputting information and a wearable vibration tactile feedback module.
[0039] As shown in FIG2 , the wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module, and a vibration feedback module, and these three modules are connected to the main control module.
[0040] The main control module uses a mainboard to connect the core board, Bluetooth module and power supply, centralizes various inputs and outputs, so that the main control module controls the three modules respectively through connecting lines and obtains status information of the multi-robot system through the Bluetooth module.
[0041] As shown in Figure 3, the main control module is installed on the main control module frame 1, and the main control module mainboard 4 is used to connect the Arduino core board 2 and the power supply, centralizing each input and output so that the main control module can control the three feedback modules respectively through the connecting line. The mainboard adopts 4 L9110S chips and 5 ULN2803 chips, which are connected to drive 4 reduction motors and 25 vibration motors through expansion interfaces 6 and 7. The mainboard comes with a Bluetooth chip interface 5, which can realize wireless control device operation through Bluetooth serial port communication. The power supply connected uses an 18650 lithium battery to provide a 7.4V stable voltage for the main control module, the extrusion force module, and the shear force module.
[0042] As shown in Figure 4, the squeeze force feedback module consists of a servo 8 and a strap (not shown). A retractable armband, used to generate squeeze tactile feedback, is connected to a reel 11. By controlling the rotation of the servo 8, varying squeeze forces are generated on the user's forearm. When both servos rotate simultaneously to tighten the strap, the armband contracts, increasing the squeeze force on the user's forearm. This device is used to provide feedback on the expansion and contraction of the human-robot formation, providing real-time feedback on the formation's size based on the force of the armband's contraction.
[0043] As shown in Figure 4, the shear force feedback module consists of a DC reduction motor 10. Four motors are fixed to the arm via printed components and straps. Friction wheels 13 are mounted on the rotating shafts. The motors drive the friction wheels to rotate, generating shear forces on the arm in four directions: forward, backward, left, and right. This is used to provide feedback on the movement direction of the human-machine formation system.
[0044] As shown in Figure 5, the vibration feedback module consists of a vibration motor array. This array of 25 flat motors is fixed to a flexible circuit board. The user secures the motors to the back of their forearms and tightens them until they feel the motors contact their skin. This provides feedback on formation changes and direction of movement during human-robot formation by providing different vibration patterns and specific orientations.
[0045] The working process and principle of the present invention are as follows:
[0046] 1. Obtain human-machine formation information in real time and transmit it to the main control module via Bluetooth wireless communication.
[0047] 2. The main control module filters out valid robot status information and classifies it based on the received information.
[0048] The human-machine formation status information can be divided into four categories: the size of the human-machine formation, the movement direction of the human-machine formation system, the formation change of the human-machine formation, and the obstacles near the human-machine formation.
[0049] 3. The main control module drives the corresponding feedback module according to the different status information of the robot.
[0050] When the size of the human-machine formation changes, the main control module drives the extrusion force feedback module to control the rotation of the servo and tighten the strap. As the human-machine formation expands, the force of the armband contraction increases; as the human-machine formation contracts, the force of the armband contraction decreases, thus providing real-time feedback on the size of the human-machine formation.
[0051] When the movement direction of the human-robot formation system changes, the main control module drives the shear force feedback module to control the standby, forward, reverse, and brake functions of the four motors. When the system issues a forward guidance command, the motor on the left side of the arm runs counterclockwise, and the motor on the right side of the arm runs clockwise, allowing the user to feel the shear force upward, providing guidance for guiding the robot forward. At the same time, the distance of the human-robot formation team relative to the target position is fed back through the motor speed.
[0052] When the formation of the human-robot team changes, the main control module drives the vibration feedback module to control the vibration motor to vibrate in an orderly manner. When the formation state of the multiple mobile robots changes from Ti to Tj, a corresponding vibration pattern is designed for each formation change state: Zd = f(Ti→Tj) (i≠j, i,j = 1, 2, ..., K);
[0053] When an obstacle appears near the human-machine formation system, the main control module drives the vibration feedback module to control the vibration of the four vibration motors on the outside of the vibration motor array, thereby providing feedback on the obstacle situation near the team; when an obstacle enters the critical distance in one direction of the team, the vibration motor in the corresponding direction will vibrate to alert the user.
[0054] It should be noted that the above content merely illustrates the technical idea of the present invention and cannot be used to limit the scope of protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications all fall within the scope of protection of the claims of the present invention.
Claims
1. A wearable tactile feedback device for human-machine formation control, characterized in that: It includes an interactive interface module and a main control module; the interactive interface module includes an input module and a wearable tactile feedback module; the wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module and a vibration feedback module, each module is worn on the arm of the user's auxiliary operator in a modular and split manner, the main control module controls the pressure feedback module, the shear force feedback module and the vibration feedback module respectively through the circuit module, and obtains the status information of the multi-robot system through the Bluetooth module.
2. A wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: The squeezing force feedback module is composed of a servo and a strap. The telescopic arm strap for generating squeezing tactile feedback is connected to the rotating part, and different squeezing forces are generated on the user's forearm by controlling the rotation of the servo. When the two servos rotate and tighten the strap at the same time, the arm strap contracts, thereby increasing the squeezing force acting on the user's forearm. This device is used to feedback the diffusion and contraction of the human-machine formation, and the size of the human-machine formation is fed back in real time through the contraction force of the arm strap.
3. A wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: The shear force feedback module is composed of a DC reduction motor; four motors are fixed to the arm through printed parts and straps, and a friction wheel is installed at the rotating shaft. The rotation of the friction wheel driven by the motor can generate shear force in the front, back, left and right directions on the arm, thereby providing feedback on the movement direction guidance of the human-machine formation system.
4. A wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: The vibration feedback module is composed of a vibration motor array; 25 vibration motors form an array fixed on a flexible circuit board, and the user fixes it on the rear end of the forearm and tightens it until the user feels the motor touching the skin; by providing different vibration modes and motor vibrations in specific directions, the formation changes and travel direction of the human-machine formation are fed back.
5. The wearable tactile feedback device for human-machine formation control according to claim 2, characterized in that: The extrusion force feedback device has the following relationship between the rotation angle of the steering gear and the applied pressure: Where N is the extrusion force, θ is the rotation angle of the servo, k is the elastic coefficient of the strap, R0 is the original radius of the strap, and R is the radius of the arm.
6. A wearable tactile feedback device for human-machine formation control according to claim 4, characterized in that: 25 vibration motors are arranged in a 5×5 array at intervals of 12 mm, fixed on a flexible PCB and directly connected to the main control module; the vibration motor is a miniature flat rotor vibration motor, and the vibration intensity of a single position can be adjusted by adjusting the input voltage; when the device is working, the output principle of the module is refreshed line by line, that is, after receiving the instruction, the main control module executes the motor vibration state control at a certain period, and generates corresponding continuous vibration feedback, which can produce a tactile sensation of motion on the human body surface.
7. The wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: The main board of the main control module uses L9110S chip and ULN2803A chip to drive 4 reduction motors and 25 vibration motors; the main board comes with a Bluetooth chip interface to realize the operation of the wireless control device through Bluetooth serial port communication; the connected power supply is powered by a small 18650 lithium battery to provide a stable voltage of 7.4V for the main control module and tactile feedback module.
8. The working process of the wearable tactile feedback device for human-machine formation control according to claim 1 is characterized by: The following steps are involved: 1) Obtain human-machine formation information in real time and transmit the human-machine formation information to the main control module through the communication module; 2) The main control module filters out and classifies the valid robot status information based on the received information; The status information of the human-machine formation can be divided into four categories, namely, the size of the human-machine formation, the movement direction of the human-machine formation system, the formation change of the human-machine formation, and the obstacles near the human-machine formation; 3) The main control module drives the corresponding feedback module according to the different status information of the robot; When the size of the human-machine formation changes, the main control module drives the extrusion module and the extrusion force feedback module to control the steering gear to rotate and tighten the straps; when the human-machine formation expands, the force of the armband contraction increases; when the human-machine formation contracts, the force of the armband contraction decreases, thereby providing real-time feedback on the size of the human-machine formation; When the movement direction of the human-machine formation system changes, the main control module drives the shear module and the shear force feedback module to control the standby, forward, reverse and brake of the four motors; when the system issues a forward guidance command, the motor on the left side of the arm runs counterclockwise and the motor on the right side of the arm runs clockwise, so that the user can feel the shear force above the arm, providing certain guidance for the user to guide the robot forward, and at the same time, the distance of the human-machine formation team relative to the target position is fed back through the motor speed; When the formation of the human-machine formation changes, the main control module drives the vibration feedback module to control the vibration motor to vibrate in an orderly manner; when the formation state of the multiple mobile robots changes from Ti to Tj, a corresponding vibration mode is designed for each formation change state: Zd=f(Ti→Tj)(i≠j,i,j=1,2,...,K); When an obstacle appears near the human-machine formation system, the main control module drives the vibration feedback module to control the four vibration motors on the outside of the vibration motor array to vibrate, thereby providing feedback on the obstacle situation near the team; when an obstacle enters the critical distance in one direction of the team, the vibration motor in the corresponding direction will vibrate to alert the user.
Citation Information
Patent Citations
Tactile feedback device and method, electronic equipment and human-computer interaction system
CN112256136A
Wearable vibration tactile motion sensing device and control method
CN113031771A
Wearable tactile feedback device for man-machine formation control
CN116627258A
System and method for providing tactile feedback
US20140176452A1
