A wearable tactile feedback device for human-machine formation control

Abstract

The present invention discloses a wearable tactile feedback device for man-machine formation control, including an interactive interface module and a main control module; the interactive interface module includes a controller module and a wearable tactile feedback module; the wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module and a vibration feedback module, and the main control module controls each module and receives control feedback instructions. The present invention can be wirelessly deployed on the operator's arm, and can provide feedback on the formation change, motion guidance, and obstacle detection of the man-machine formation by generating three tactile feedback signals of extrusion, shearing, and vibration. It is more advantageous to provide feedback to the wearer on the perception ability and task status of the external limbs based on the tactile channel, which reduces the difficulty of human-machine interaction and creates a highly immersive operating experience for users.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-machine interaction, and in particular relates to a wearable tactile feedback device for human-machine formation control. Background Art

[0002] The research on human-robot formation systems has attracted more and more attention from scientific researchers in recent years. The human-robot formation system, which is composed of multiple robots with relatively simple structures and functions, can complete tasks that are difficult for a single robot to complete through the complementary capabilities and coordinated actions among its members, and has the advantages of good robustness and scalability.

[0003] At present, visual feedback is a common way to convey human-machine teaming information to users. However, in actual operation scenarios, due to the increase in the number of robots and the high load of the visual feedback channel and the occlusion of the viewing angle, users may not be able to get effective feedback in the visual channel. Tactile feedback has its unique advantages over other sensory channels. Human touch is not only the most robust and widely distributed sensory channel for humans, but also has the characteristics of sensitivity, bidirectionality and privacy. Therefore, compared with visual and auditory channels, the tactile channel has more advantages in providing feedback to the wearer on the understanding ability and task status of the external limbs, and can play a unique role in reducing the difficulty of human-machine interaction and creating a highly immersive operation experience for users. Summary of the invention

[0004] To solve the above problems, the present invention discloses a wearable tactile feedback device for human-machine formation control, which adopts three real-time information feedback mechanisms based on the tactile feedback principle. The tactile feedback information generated by the device can serve as an effective supplement to the operator's visual and auditory perception, thereby improving the operator's ability to perceive the surrounding environment and the current formation situation, and providing directional guidance to guide the operator to quickly locate the mission target in a complex environment.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A wearable tactile feedback device for human-machine formation control, comprising an interactive interface module and a main control module;

[0007] The interactive interface module includes an input module and a wearable tactile feedback module;

[0008] The wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module and a vibration feedback module, wherein the vibration feedback module is directly connected to the main control module, and the shear force feedback module and the extrusion force feedback module are uniformly connected to the main control module through a cable;

[0009] The main control module uses a mainboard to connect the core board, Bluetooth module and power supply, centralizes various inputs and outputs, so that the main control module controls the three modules respectively through the connecting lines, and obtains the status information of the multi-robot system through the Bluetooth module. The mainboard uses L9110S chip and ULN2803 chip to drive 4 reduction motors and 25 vibration motors. The mainboard is equipped with a Bluetooth chip interface, and the wireless control device is realized through Bluetooth serial port communication; the connected power supply is powered by 18650 lithium battery, which provides 7.4V stable voltage for the main control module and tactile feedback module.

[0010] The squeezing force feedback module is composed of a steering gear and a strap. The telescopic arm strap for generating squeezing tactile feedback is connected to the rotating part. The rotating part is driven to rotate by controlling the rotation of the steering gear. While controlling the tightness of the strap, different squeezing forces are generated on the user's forearm. The rotation angle of the steering gear and the applied pressure have the following relationship:

[0011]

[0012] Where N is the squeezing force, θ is the rotation angle of the servo, k is the elastic coefficient of the strap, R0 is the original radius of the strap, and R is the arm radius. When the two servos rotate and tighten the strap at the same time, the armband contracts, increasing the squeezing force on the user's forearm. This device is used to feedback the spread and contraction of the human-machine formation, and the size of the human-machine formation is fed back in real time through the contraction force of the armband.

[0013] The shear force feedback module is composed of a DC reduction motor. Four motors are fixed on the arm through printed parts and straps. A friction wheel is installed on the shaft. The rotation of the friction wheel driven by the motor can generate shear force in four directions: front, back, left and right on the arm, so as to provide feedback on the movement direction of the human-machine formation system.

[0014] The vibration feedback module is composed of a vibration motor array. 25 flat motors are formed into an array and fixed on a flexible circuit board. The user fixes it on the rear end of the forearm and tightens it until the motor touches the skin. By providing different vibration modes and motor vibrations in specific directions, the formation changes and travel directions of the human-machine formation are fed back.

[0015] In order to realize tactile feedback of human-machine formation control, the design principle of the present invention is:

[0016] 1. Design using three forms of tactile feedback

[0017] Tactile feedback has its unique advantages over other sensory channels. Human touch is not only the most robust and widely distributed sensory channel for humans, but also has the characteristics of being sensitive, bidirectional and private. The wearable device can generate three types of tactile forces on the user's skin surface, providing the user with four types of feedback information on the control of the human-machine formation system, making up for some of the shortcomings of visual feedback in practical applications. This system can effectively help users perceive the surrounding environment and the situation of the human-machine formation, and provide directional guidance to guide users to quickly locate the mission target position.

[0018] Feedback information 1, contraction and expansion of the human-machine formation: This feedback information is executed through the squeeze force module. The contraction scale of the system corresponds to the scale of the servo rotation. When the user changes the size of the formation, the system immediately sends the corresponding servo degree to the squeeze module. The two servos work simultaneously to contract the straps, so that the user can feel the squeeze force generated by the contraction of the straps.

[0019] Feedback information 2, task direction guidance: This feedback information is executed through the shear force module. The shear module will provide real-time feedback on the relative direction and relative distance between the parent / child target point and the operator, so as to guide the operator to the target point in sequence. The difference in relative direction between the operator and the parent / child target will cause the shear module to generate shear forces in different directions, and the difference in relative distance will cause the rotation speed of the reduction motor in the shear module to be different, resulting in different shearing forces.

[0020] Feedback information three, human-machine formation change information. Through the vibration array module, different shapes are drawn on the operator's arm using the tactile illusion phenomenon of apparent motion. Apparent motion means that if there are some discrete vibration contacts on the surface of human skin that vibrate according to a certain vibration duration and start interval, the user's skin surface will produce a feeling of continuous vibration.

[0021] Feedback information 4: obstacle direction information: through the vibration motors around the vibration array module, when the robot in the corresponding direction of a certain formation detects that an obstacle has entered the critical distance, the vibration motors in the corresponding direction will generate vibrations in the corresponding direction to remind the operator that the robot formation has encountered an obstacle. This helps the operator to perceive the moving status of the human-machine formation behind him while focusing on the front.

[0022] 2. Lightweight modular design

[0023] The main control device is integrated into one, and each module can be worn on the user's co-operator's arm (usually the left hand) in a split type. The user can easily put it on and take it off. Just turn on the power to connect to the human-machine formation system and start running. The split-based design helps to reduce the interference of different tactile feedback and improve the user's efficiency in distinguishing different feedback. When wearing this device, the user can still use the wearing hand normally to perform various tasks and pass the test of the simulation environment.

[0024] 3. Use self-designed circuits and realize communication

[0025] The circuit module connection mechanism is designed to connect the single-chip core board and each module. The circuit design includes the driver chip of each module motor, which can directly drive the motor of each module. At the same time, a Bluetooth communication module is provided, which can directly communicate with the single-chip chip through a wireless serial port to control each module to respond to the corresponding situation. Since this device has low power consumption, it can be powered by a small lithium battery module, which has a long battery life while being miniaturized.

[0026] The beneficial effects of the present invention are:

[0027] (1) The present invention adopts a real-time information feedback mechanism for human-machine formation based on three tactile feedback principles. The tactile feedback information generated by the mechanism can serve as an effective supplement to the operator's visual and auditory perception, thereby improving the operator's ability to perceive the surrounding environment and the current formation situation, and providing directional guidance to guide the operator to quickly locate the mission target in a complex environment.

[0028] (2) The present invention adopts a small wireless wearable design. The main control device is integrated into one body, and each module can be worn on the arms of the operator and the assistant operator in a separate manner. The operator can put it on and take it off very conveniently. The operator only needs to turn on the power to connect to the human-machine formation system and start operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a schematic diagram of the working state of the test device system according to an embodiment of the present invention;

[0030] Figure 2 It is the hardware connection diagram of the device;

[0031] Figure 3 This is the exploded structure diagram of the main control module;

[0032] Figure 4 It is the structural diagram of the extrusion force module and the shear force module;

[0033] Figure 5 This is the structural diagram of the vibration array module.

[0034] List of Figure Symbols:

[0035] 1. Main control module frame; 2. Arduino Mega2560 core board; 3. Power supply interface of main control module; 4. Main control module mainboard; 5. Bluetooth module interface reserved on the mainboard; 6. Extrusion force feedback module and shear force feedback module expansion interface; 7. Vibration feedback module interface; 8. 20Kg servo; 9. Extrusion force feedback module and shear force feedback module expansion board; 10. Gear motor; 11. Tape wheel for applying extrusion force; 12. Bearing; 13. Friction wheel for shear force feedback, 14. Shear force feedback module in the front and rear directions. DETAILED DESCRIPTION

[0036] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0037] The present invention designs a wearable device with three forms of tactile feedback based on a multi-mobile robot collaborative control system, and uses a VR controller as a command input interface for the collaborative control of the human-machine formation; at the same time, it combines multiple force tactile information to provide feedback on the motion state of the human-machine formation.

[0038] like Figure 1 As shown, it includes an interactive interface module, a communication module and a main control module. The interactive interface module includes a VR controller module for inputting information and a wearable vibration tactile feedback module.

[0039] like Figure 2 As shown, the wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module and a vibration feedback module, wherein the vibration feedback module is directly connected to the main control module, and the shear force feedback module and the extrusion force feedback module are uniformly connected to the main control module through a cable.

[0040] The main control module uses a mainboard to connect the core board, the Bluetooth module and the power supply, centralizes various inputs and outputs, so that the main control module controls the three modules respectively through connecting lines, and obtains the status information of the multi-robot system through the Bluetooth module.

[0041] like Figure 3 As shown, the main control module uses a self-designed motherboard to connect the core board and the power supply, centralizing each input and output, so that the main control module can control the three feedback modules separately through the connecting wires. The mainboard uses 4 L9110S chips and 5 ULN2803 chips to drive 4 reduction motors and 25 vibration motors. The mainboard comes with a Bluetooth chip interface, and the wireless control device can be operated through Bluetooth serial port communication. The connected power supply is powered by a 18650 lithium battery to provide a stable voltage of 7.4V for the main control module and the tactile feedback module.

[0042] like Figure 4As shown, the squeezing force feedback module is composed of a servo and a strap. The telescopic armband used to generate squeezing tactile feedback is connected to the rotating part. The rotating part is driven to rotate by controlling the rotation of the servo. While controlling the tightness of the strap, different squeezing forces are generated on the user's forearm. When the two servos rotate and tighten the strap at the same time, the armband contracts, increasing the squeezing force on the user's forearm. This device is used to feedback the spread and contraction of the human-machine formation, and the size of the human-machine formation is fed back in real time through the contraction force of the armband.

[0043] like Figure 4 As shown, the squeezing force feedback module is composed of a servo and a strap. The telescopic armband for generating squeezing tactile feedback is connected to the rotating part, and different squeezing forces are generated on the user's forearm by controlling the rotation of the servo. When the two servos rotate and tighten the strap at the same time, the armband contracts, increasing the squeezing force on the user's forearm. This device is used to feedback the spread and contraction of the human-machine formation, and the size of the human-machine formation is fed back in real time through the contraction force of the armband.

[0044] like Figure 4 As shown, the shear force feedback module is composed of a DC reduction motor. Four motors are fixed on the arm through printed parts and straps, and a friction wheel is installed on the shaft. The rotation of the friction wheel driven by the motor can generate shear force in the front, back, left and right directions on the arm, so as to provide feedback on the movement direction guidance of the human-machine formation system.

[0045] like Figure 5 As shown in the figure, the vibration feedback module is composed of a vibration motor array. 25 flat motors form an array fixed on a flexible circuit board. The user fixes it on the rear end of the forearm and tightens it until the motor touches the skin. By providing different vibration modes and motor vibrations in specific directions, the formation changes and travel directions of the human-machine formation are fed back.

[0046] The working process and principle of the present invention are as follows:

[0047] 1. Obtain human-machine formation information in real time and transmit it to the main control module via Bluetooth wireless communication.

[0048] 2. The main control module filters out and classifies the valid robot status information based on the received information.

[0049] The human-machine formation status information can be divided into four categories: the size of the human-machine formation, the movement direction of the human-machine formation system, the formation change of the human-machine formation, and the obstacles near the human-machine formation.

[0050] 3. The main control module drives the corresponding feedback module according to the different status information of the robot.

[0051] When the size of the human-machine formation changes, the main control module drives the extrusion force feedback module to control the steering gear to rotate and tighten the straps; when the human-machine formation expands, the force of the armband contraction increases; when the human-machine formation contracts, the force of the armband contraction decreases, thereby achieving real-time feedback on the size of the human-machine formation;

[0052] When the movement direction of the human-machine formation system changes, the main control module drives the shear force feedback module to control the standby, forward, reverse and brake of the four motors; when the system issues a forward guidance command, the motor on the left side of the arm runs counterclockwise and the motor on the right side of the arm runs clockwise, so that the user can feel the shear force above the arm, providing certain guidance for the user to guide the robot forward, and at the same time, the distance of the human-machine formation team relative to the target position is fed back through the motor speed;

[0053] When the formation of the human-machine formation changes, the main control module drives the vibration feedback module to control the vibration motor to vibrate in an orderly manner; when the formation state of the multiple mobile robots changes from Ti to Tj, a corresponding vibration mode is designed for each formation change state: Zd=f(Ti→Tj)(i≠j,i,j=1,2,...,K);

[0054] When an obstacle appears near the human-machine formation system, the main control module drives the vibration feedback module to control the four vibration motors on the outside of the vibration motor array to vibrate, thereby providing feedback on the obstacle situation near the team; when an obstacle enters the critical distance in one direction of the team, the vibration motor in the corresponding direction will vibrate to alert the user.

[0055] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A wearable tactile feedback device for human-machine formation control, characterized in that: It includes an interactive interface module and a main control module; the interactive interface module includes an input module and a wearable tactile feedback module; the wearable tactile feedback module includes an extrusion force feedback module, a shear force feedback module and a vibration feedback module, each of which is worn on the arm of the user's assistant operator in a modular and split manner, and the main control module controls the pressure feedback module, the shear force feedback module and the vibration feedback module respectively through the circuit module, and obtains the status information of the multi-robot system through the Bluetooth module; The squeezing force feedback module is composed of a steering gear and a strap. The telescopic arm strap for generating squeezing tactile feedback is connected to the rotating part, and different squeezing forces are generated on the user's forearm by controlling the rotation of the steering gear. When the two steering gears rotate and tighten the strap at the same time, the arm strap contracts, thereby increasing the squeezing force acting on the user's forearm. This device is used to feedback the spread and contraction of the human-machine formation, and the size of the human-machine formation is fed back in real time through the contraction force of the arm strap. The shear force feedback module is composed of a DC reduction motor; four motors are fixed on the arm through printed parts and straps, and a friction wheel is installed at the rotating shaft. The motor drives the friction wheel to rotate to generate shear force in four directions: front, back, left and right on the arm, so as to provide feedback on the movement direction guidance of the human-machine formation system; The vibration feedback module is composed of a vibration motor array; 25 vibration motors form an array fixed on a flexible circuit board, and the user fixes it on the rear end of the forearm and tightens it until the user feels the motor touching the skin; by providing different vibration modes and motor vibrations in specific directions, the formation changes and travel direction of the human-machine formation are fed back.

2. A wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: In the extrusion force feedback module, the rotation angle of the servo and the applied pressure have the following relationship: Where N is the extrusion force, θ is the rotation angle of the servo, k is the elastic coefficient of the strap, R0 is the original radius of the strap, and R is the radius of the arm.

3. A wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: 25 vibration motors are arranged in a 5×5 array at intervals of 12 mm, fixed on a flexible PCB and directly connected to the main control module; the vibration motor is a miniature flat rotor vibration motor, and the vibration intensity of a single position can be adjusted by adjusting the input voltage; when the device is working, the output principle of the module is refreshed line by line, that is, after receiving the instruction, the main control module executes the motor vibration state control at a certain period, and generates corresponding continuous vibration feedback, which can produce a tactile sensation of motion on the human body surface.

4. A wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: The main board of the main control module uses L9110S chip and ULN2803A chip to drive 4 reduction motors and 25 vibration motors; the main board comes with a Bluetooth chip interface to realize the operation of the wireless control device through Bluetooth serial port communication; the connected power supply is powered by a small 18650 lithium battery to provide a stable voltage of 7.4V for the main control module and tactile feedback module.

5. The tactile feedback method of a wearable tactile feedback device for human-machine formation control according to claim 1, characterized in that: The following steps are involved: 1) Obtain human-machine formation information in real time and transmit the human-machine formation information to the main control module through the communication module; 2) The main control module filters out and classifies the valid robot status information based on the received information; The human-machine formation status information is divided into four categories, namely, the size of the human-machine formation, the movement direction of the human-machine formation system, the formation change of the human-machine formation, and the obstacles near the human-machine formation; 3) The main control module drives the corresponding feedback module according to different status information of the robot; When the size of the human-machine formation changes, the main control module drives the extrusion force feedback module to control the steering gear to rotate and tighten the straps; when the human-machine formation expands, the force of the armband contraction increases; when the human-machine formation contracts, the force of the armband contraction decreases, thereby achieving real-time feedback on the size of the human-machine formation; When the movement direction of the human-machine formation system changes, the main control module drives the shear force feedback module to control the standby, forward, reverse and brake of the four motors; when the system issues a forward guidance command, the motor on the left side of the arm runs counterclockwise and the motor on the right side of the arm runs clockwise, so that the user can feel the shear force above the arm, providing certain guidance for the user to guide the robot forward, and at the same time, the distance of the human-machine formation team relative to the target position is fed back through the motor speed; When the formation of the human-machine formation changes, the main control module drives the vibration feedback module to control the vibration motor to vibrate in an orderly manner; when the formation state of the multiple mobile robots changes from Ti to Tj, a corresponding vibration mode is designed for each formation change state: Zd=f(Ti→Tj)(i≠j,i,j=1,2,...,K); When an obstacle appears near the human-machine formation system, the main control module drives the vibration feedback module to control the four vibration motors on the outside of the vibration motor array to vibrate, thereby providing feedback on the obstacle situation near the team; when an obstacle enters the critical distance in one direction of the team, the vibration motor in the corresponding direction will vibrate to alert the user.

Images