Traction-type guide leg-arm robot

Abstract

The present invention discloses a traction-type four-legged blind-guiding leg-arm robot, comprising: a blind-guiding leg-arm robot body; legs and feet of the blind-guiding leg-arm robot, the legs and feet being used to support the robot body; a mechanical arm, the blind-guiding leg-arm robot being connected to the mechanical arm; and the blind-guiding leg-arm robot guiding the blind through the mechanical arm. The traction-type blind-guiding leg-arm robot of the present invention has better adaptability to complex outdoor environments, and the leg-arm coordination control algorithm enables the robot to have better stability during walking; the composite scene perception module enables the robot to perform three-dimensional reconstruction and environmental information recognition in real time; the positioning module enables the robot to perform global positioning of man-machine in real time; the blind person posture tracking module enables the robot to track the blind person in real time, measure the blind person's posture, and detect the rear / lateral environment; the tactile prompt module can feed back step information to the blind person, making the interaction between the robot and the blind person more convenient and humane, and realizing autonomous guidance.

Description

Technical Field

[0001] The invention relates to a blind-guiding leg-arm robot system and a blind-guiding method thereof, and belongs to the field of auxiliary equipment. Background Art

[0002] The visually impaired are a vulnerable group in society. However, there is a shortage of professional caregivers and backward assistive devices. The task of providing travel, medical care, education, employment and other services to the blind is arduous and far from meeting their needs for normal life.

[0003] In recent years, scholars from many countries have developed various types of blind aids and have made gratifying progress. However, overall, there are defects such as poor compliance with the blind and low environmental adaptability. They are far from being recognized and promoted by the market, and there is still a lot of room for improvement in both breadth and depth. The reasons are: ① The interactive information is inaccurate and too complicated, which makes it difficult for the blind to handle and has a heavy psychological burden; ② It is difficult to provide the blind with a sense of security at the psychological level. Surveys show that the blind are still more accustomed to traveling with guide dogs and people. However, the current shortage of guide dogs and accompanying personnel is difficult to change in a short period of time.

[0004] At present, guide devices can be roughly divided into three categories: smart guide sticks, wearable guide devices and mobile guide robots. The first two categories usually do not have a power drive mechanism. Their characteristics are to provide auditory / tactile feedback of environmental and navigation information to the blind to assist them in walking independently. This alone is not enough to satisfy the blind's sense of security, and neither of these two methods is suitable for blind people to travel. Mobile guide robots use guiding mechanisms to autonomously pull the blind to walk. The use of a robot-first, blind-following guide method can greatly improve walking safety, so it has received widespread attention in recent years. At the current level of technology, compared with smart guide sticks and wearable guide devices, mobile guide robots have the advantages of high reliability and high safety. The traction method can greatly reduce the psychological burden of the blind and is more easily accepted by the blind.

[0005] Although the achievements of guide robots are gratifying, there are still some problems. For example, Chinese patent CN111609851A discloses a mobile guide robot system and a guide method, but does not take into account the terrain adaptability in complex outdoor environments, such as complex terrains such as potholes and steps; Chinese patent CN113304019A discloses a six-legged guide robot, but it lacks an active compliance mechanism between the blind and the blind, and the stiffness of the human-machine connection mechanism may be improper, resulting in uncoordinated walking and severe "pulling". In response to the above problems, a guide leg-arm robot is needed. Through its special mechanical structure design and motion control algorithm, the leg-arm robot can achieve stable walking and adaptability to complex outdoor environments. The robot actively complies with the blind through the mechanical arm, making the human-machine more coordinated. Combined with visual / force feedback, it can complete human-machine interaction functions such as navigation, obstacle avoidance, and road information recognition, so as to better guide the blind to travel. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a traction-type guide leg-arm robot to solve the problems of existing guide robots such as lack of adaptation mechanism for the blind and weak environmental adaptability, and to provide a guide robot that is more convenient for visually impaired people to travel and live their daily lives.

[0007] To achieve the above-mentioned purpose, the present invention provides a traction-type guiding leg-arm robot, comprising: a guiding leg-arm robot body; legs and feet of the guiding leg-arm robot, wherein the legs and feet are used to support the robot body; a mechanical arm, and the guiding leg-arm robot is connected to the mechanical arm; the guiding leg-arm robot guides the blind through the mechanical arm.

[0008] The advantages of the traction-type blind-guiding leg-arm robot of the present invention are that the leg-arm structure enables the robot to have stronger adaptability to complex outdoor environments, the leg-arm coordination control algorithm enables the robot to have better stability during walking, the composite scene perception module enables the robot to perform three-dimensional reconstruction and environmental information recognition in real time, the positioning module enables the robot to perform global positioning of man-machine in real time, the blind posture tracking module enables the robot to track the blind in real time, measure the blind posture and detect the rear / lateral environment, plan the motion trajectory of the end of the traction arm according to the perceived environment and image information, adjust the position of the robot trunk, and safely pull the blind to the desired relative posture; the tactile prompt module can feedback the step information to the blind, making the interaction between the robot and the blind more convenient and humanized, the robot and the blind interact through the mechanical arm force perception, and actively comply with the blind mechanism, which is more in line with the blind's operating habits, and the visual interaction between the robot and the surrounding environment improves safety. In short, the traction-type blind-guiding leg-arm robot can bring convenience to the blind's travel and normal life.

[0009] Compared with the prior art, the advantages of the present invention are:

[0010] 1. In order to alleviate the "pulling" phenomenon on the traction mechanism, the present invention proposes a rigid-flexible coupling active traction arm based on ergonomics and system bionics, which can actively adapt to the movement tendency of the blind, guide the blind to pass through obstacles safely, and meet the special needs of traction and guidance for the blind for safety and comfort.

[0011] 2. The present invention proposes a method for assessing safe passage of humans and machines in complex environments. By fusion of 3D point clouds and near-field semantic images at the feature level, a method for judging the transparency of a passable window path is proposed. The high-open passage area adjacent to the blind path is quickly extracted, and the danger level value is calculated based on the near-field environmental information and the human-machine spatial configuration to form a comprehensive assessment of the passability of the local area. Finally, a safe walking path is generated based on the waypoint distance adjustment strategy of the collision point.

[0012] 3. The present invention has a tactile prompt module. The outdoor environment is noisy and voice prompts may cause the blind to fail to receive obstacle information in time. The present invention integrates a micro-vibration plate on the U-shaped handle of the traction arm. When it is inevitable to pass through steps, the pulse length and intensity are adjusted to feed back the step information to the blind to guide them to pass safely.

[0013] 4. The present invention adopts a guiding method in which the robot goes first and the blind follows, which greatly improves the efficiency and safety of guiding the blind and makes travel more convenient for the blind. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 1 is a hardware structure diagram of the traction-type blind-guiding leg-arm robot according to an embodiment of the present invention;

[0015] Figure 2 is a schematic diagram of a traction-type blind-guiding leg-arm robot according to an embodiment of the present invention;

[0016] Figure 3 is a control architecture diagram of the traction-type blind-guiding leg-arm robot according to an embodiment of the present invention;

[0017] Figure 4 is a control system structure diagram of the traction-type blind-guiding leg-arm robot in a constrained space according to an embodiment of the present invention;

[0018] Among them: 1. Robot legs and feet; 2. Robot body 3. Floating base; 4. Traction arm; 5. Flexible joint; 6. LiDAR; 7. RGB-D camera; 8. Force sensor; 9. Side camera; 10. Mobile tracking tower; 11. U-shaped grip; 12. Vibration module; DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. The specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. For ease of description, only some structures related to the present invention are shown in the accompanying drawings.

[0020] Embodiment 1

[0021] like Figure 1 , Figure 2 As shown, in order to achieve the above purpose, the present invention proposes the following solutions:

[0022] In the following content, "the blind-guiding leg-arm robot" refers specifically to the present invention. A rigid-flexible coupled active traction arm is designed, combined with a quadruped robot platform, and integrated with a blind-guiding leg-arm robot to assist the blind in walking.

[0023] The overall structure of the present invention is:

[0024] The leg and foot 1 of the guide robot leg-arm robot supports the guide robot body 2. A battery, a driver, a controller, and a control module are installed in the robot body 2. A floating base 3 is installed on the back of the robot body 2 for connecting with the traction arm 4. A laser radar 6 and a depth camera 7 are installed on the front of the body 2, a mobile tracking tower 10 is installed on the rear of the body 2, and a camera is installed on the side of the body 2.

[0025] The traction arm 4 of the guide robot is installed on the back of the robot body 2. Its flexible end contacts the blind person through a spring and a soft material U-shaped handle. The U-shaped handle contains a vibration module. When passing through the stairs, the armature is used to adjust the pulse length and intensity of the vibration piece to feed back the step information to the blind person.

[0026] The number of legs 1 of the guiding leg-arm robot described in the present invention is 4, which are respectively installed on the left front, left rear, right front and right rear of the body 2.

[0027] The guiding leg-arm robot of the present invention adopts the leg-foot 1 structure to walk, and can pass through complex road conditions, such as steps, potholes, barrier-free passages, traffic intersections and other common outdoor scenes.

[0028] The blind-guiding leg-arm robot of the present invention adopts a leg-foot 1 structure to support a robot body 2 .

[0029] The blind-guiding leg-arm robot of the present invention adopts the leg-foot 1 structure to move, and can complete forward, backward, left, right and left movements and steering movements.

[0030] The guiding blind leg-arm robot body 2 of the present invention is supported by the robot legs and feet 1, and other functional modules of the guiding blind leg-arm robot are carried on the body.

[0031] The guiding leg-arm robot of the present invention adopts a traction arm 4 connected to a robot body 2 through a floating base 3. The traction arm 4 and the robot body 2 retain the degree of freedom of rotation to adapt to the contact angles of turning and different road surfaces.

[0032] The U-shaped handle 11 at the upper end of the traction arm 4 of the guide leg-arm robot of the present invention directly contacts and interacts with the blind. The U-shaped handle 11 is equipped with a vibration module. When passing through the stairs, the armature is used to adjust the pulse length and intensity of the vibration piece, and the step information is fed back to the blind to guide them to pass safely.

[0033] The guiding leg-arm robot traction arm 4 of the present invention is provided with a flexible joint 5 and a force sensor 8. The blind person interacts with the robot by traction of the mechanical arm in a force interaction manner, thereby completing operation commands such as movement, rotation, and speed control of the robot.

[0034] The traction arm 4 is lightweight and optimized. During the traction and guidance process, the blind person perceives the interaction force to make decisions, while the robot needs to overcome the negative effects of the interaction force. If there is no flexible joint 5 between the human and the machine, periodic impacts will occur during the traction process, making the robot and the blind person walk uncoordinated. The active traction arm with a flexible unit can actively adapt to the blind person, solve problems such as improper stiffness of the human-machine connection mechanism, and effectively alleviate the "pulling" phenomenon.

[0035] The composite scene perception module of the guide leg-arm robot of the present invention includes a laser radar, an RGB-D camera, a force sensor and a lateral camera, etc. It is used to perceive the external environment in real time, process environmental information, and detect and evaluate the safe passage of man-machine. The laser radar 6 and the RGB-D camera 7 are placed in front of the robot body. First, the 3D point cloud data of the surrounding environment is obtained by scanning to realize the scene reconstruction of the far field of view; secondly, the 3D point cloud and the near field of view semantic image are feature-level fused using the confidence map, and a method for judging the transparency of the passable window path based on depth information is constructed to quickly extract the high-openness passage area adjacent to the blind path; furthermore, a hazard level function that integrates ground features (potholes, road curvature, steps) and obstacles is designed, and the hazard level value is calculated according to the extracted near field of view environmental information and the man-machine space configuration to form a comprehensive evaluation of the passability of the local area; finally, a smooth safe walking path is generated based on the waypoint away adjustment strategy of the collision point, and the corresponding man-machine posture and gait adjustment is made through the upper-level planner of visual / force feedback.

[0036] The blind person posture tracking module of the guiding leg-arm robot of the present invention comprises a mobile tracking tower 10 equipped with a depth camera. The mobile tracking tower 10 is placed behind the robot body and is used for real-time tracking of the blind person, measuring the blind person's posture and detecting the rear / lateral environment to support the human-machine posture adjustment of visual servoing.

[0037] In the blind guiding task, an important issue is how to obtain the human body's motion intention. If the robot knows the human body's motion intention and the human body's stiffness, it can more conveniently design an impedance controller for efficient human-machine interaction. In the blind guiding task, it is hoped that the robot can be compliant with the blind and complete the task with similar behaviors. If the human-machine has different behaviors during the blind guiding process, it will lead to inefficient or unsuccessful tasks. Similar behaviors mean that the human-machine should have similar initial positions and motion target positions, and the human's stiffness matrix should be the same as the robot's stiffness matrix.

[0038] The guiding leg-arm robot positioning module of the present invention is composed of an IMU and a Beidou navigator, and performs global positioning of a person and a machine.

[0039] The working principle of the present invention is:

[0040] The guide leg-arm robot controls forward, backward, left, right or rotation by periodically lifting and swinging the legs of leg foot 1; and passes through different complex terrains, such as flat ground, stairs, potholes, traffic signal intersections, etc., by using leg foot 1 at different lifting heights.

[0041] The guide leg-arm robot uses laser radar to scan the surrounding environment and obtain 3D point cloud information of the surrounding environment. It uses a depth camera to identify the road surface, obstacles, pedestrians, traffic intersections, etc. Through the hazard level function of the comprehensive ground features and obstacles, the hazard level value is calculated according to the extracted near-field environment information and the human-machine space configuration, forming a comprehensive assessment of the passability of the local area.

[0042] The guide leg-arm robot collects the blind person's voice commands through the voice module, converts them into robot control commands through a series of algorithms, and changes the robot's movement mode through force interaction. Through force sensing, the blind can know whether there are obstacles in the environment.

[0043] The guiding leg-arm robot complies with the blind person through the traction arm 4 to complete the guiding function. During the guiding process, the collected environmental information forms a local passable evaluation, and finally generates a smooth and safe walking path based on the waypoint away from the collision point adjustment strategy, and makes corresponding human-machine posture and gait adjustments through the upper-level planner of visual / force feedback. The blind person holds the U-shaped handle 11, and by applying different forces, the robot senses the size and direction of the blind person's force, thereby responding to the blind person's intentions, such as speed, start, stop, etc.

[0044] The U-shaped handle 11 of the traction arm of the guide leg-arm robot is equipped with a vibration module. When passing through the stairs, the armature is used to adjust the pulse length and intensity of the vibration piece to feed back the step information to the blind.

[0045] The guide leg-arm robot can combine the information collected by lidar and depth cameras with GPS to meet the daily travel needs of the blind.

[0046] In the process of guiding the blind, the guide leg-arm robot changes in a variety of open / closed chain motion structures alternately. Based on the hierarchical recursive control idea and the desired trunk posture, the output of the virtual controller is obtained through the trunk dynamics model, combined with the trajectory planning of the leg-arm end, and then the joint control torque is solved based on the single leg / arm dynamics model in parallel.

[0047] The human-machine collaborative gait generation strategy of the present invention firstly designs a compliant collaborative gait evaluation function according to the structural characteristics of the leg-arm robot; secondly, the state information such as the center of mass velocity and interaction force of the robot body is input into the self-learning network, and an auxiliary gradient loss function is constructed based on the approximate strategy optimization theory; thirdly, the trust region optimization theory is integrated to adaptively constrain the strategy update, adjust the iteration step size in the training process, and improve the sample utilization rate; finally, the learning rate is dynamically adjusted based on the relative entropy principle in information theory, so that the algorithm can quickly converge to a gait that adapts to the current traction and blindness while ensuring the stability of the strategy network, thereby realizing a model-free direct mapping of state information to collaborative gait.

[0048] The human-machine coordinated motion control method of the present invention firstly designs a rollover moment optimizer to determine the desired posture of the trunk according to the human-machine interaction force measurement value, the human-machine relative posture and the environmental information; secondly, based on the dynamic model of the single rigid body of the trunk and inverse kinematics, the desired trajectory of the trunk is converted into the output of five virtual controllers, and after selecting the gait from the gait library, the terminal motion trajectory of the traction arm, the swing foot and the supporting foot is planned; finally, according to the output of the five virtual actuators, for the dynamic model of a single leg / arm, the closed-chain leg-arm terminal position constraint, the traction arm and the blind manpower constraint, the swing leg and the supporting leg terminal motion constraint, the foot end friction cone constraint, the traction space constraint and the joint limit constraint are considered as a whole, and a multi-objective optimization method based on differential dynamic programming and alternating directions of multipliers is proposed, and the dynamic model of the leg / arm is solved in parallel to generate the joint torque that meets all the constraints, so as to realize the robot's compliant traction blind operation.

[0049] Visual servo traction control framework for posture adjustment: includes impedance control module, impedance parameter optimization module, Bayesian human motion intention prediction module, position control module, and traction control module; first, the Bayesian human motion intention prediction module will estimate the human motion intention and human impedance parameters, and transmit the information to the impedance control module and the impedance parameter optimization module. After the impedance parameters are learned, they are transmitted back to the Bayesian human motion intention prediction module. The impedance control module is connected with the position control module and the traction control module to realize the constraint of the subjective movement intention of the robot arm and to pull the blind with appropriate force.

[0050] The visual servo traction control design for posture adjustment of the blind in the present invention first uses the quadruped robot as a stable operating platform of the traction arm, determines the robot leg configuration according to the perceived environment, and adjusts the robot torso position; secondly, establishes a mapping relationship between the image space and the joint space of the traction arm, realizes depth information decoupling based on the depth independent interaction matrix, and plans the motion trajectory of the end of the traction arm locally according to the image information; finally, based on the Bayesian estimation learning impedance parameters of the traction arm dynamics model and the human-machine force interaction model, designs the impedance control strategy of the human in the loop, and safely tows the blind to the desired relative posture.

[0051] In the blind guide operation, the robot goes first and the blind person follows based on the navigation of the blind path. In narrow spaces or turns, the robot needs to pull the blind person to the side of the robot. To ensure the safety of interaction, the end effector of the robot system should always remain in the restricted task space to prevent the operation radius from being too large and causing accidents.

[0052] In order to ensure the safety of human-machine interaction, the end effector of the robot arm should always remain in the restricted task space. First, the reference trajectory is shaped to ensure that the reference motion trajectory is within the restricted task space.

[0053] The high-precision target detection and recognition in the present invention firstly designs a lightweight baseline network combined with an attention mechanism to extract specific semantic information of walkable areas such as blind paths and barrier-free passages; secondly, constructs a saliency space pyramid pooling module to downsample and perform multi-scale semantic segmentation on the walkable area, uses pooling operations of different sizes to encode the extracted specific semantic information, and obtains deep features through upsampling; thirdly, adopts a spatial detail preservation module to extract shallow features containing other environmental information such as pedestrians and road signs; finally, builds a deep feature fusion module to combine the obtained deep and shallow semantic features to achieve "end-to-end" high-precision recognition of environmental information.

[0054] The comprehensive evaluation method for safe passage of man and machine in the present invention firstly realizes scene reconstruction of far field based on regional clustering 3D point cloud segmentation algorithm; secondly, feature-level fusion of 3D point cloud and near field semantic image is performed using confidence map, and a passable window path transparency judgment method based on depth information is constructed to quickly extract high openness passage area adjacent to the blind path; thirdly, a hazard level function is designed that integrates ground features (potholes, road curvature, steps) and obstacles, and the hazard level value is calculated according to the extracted near field environment information and man-machine spatial configuration to form a comprehensive passability evaluation of the local area; finally, a smooth safe walking path is generated based on the waypoint away adjustment strategy of the collision point, and the corresponding man-machine posture and gait adjustment is made through the upper-level planner of visual / force feedback.

[0055] Algorithm 1: Bayesian Human Intention Estimation

[0056] The dynamics of the manipulator system in the task space is: Human dynamics model: It can be simplified into a spring model in the task space: f h =K h (x h -x)

[0057] Among them, f e ∈R n represents external force, f∈R n Represents the control input of the robot arm, K h ∈R h×h represents the stiffness matrix of the person, x h ∈R h Represents human motion intention, x∈R h is the actual position, f h ∈R h Represents the force vector between a person and the object they are touching.

[0058] Design impedance model: Among them, M d ,D d ,K d They correspond to the inertia coefficient, damping coefficient and stiffness coefficient respectively, collectively referred to as impedance parameters. In low-speed human-computer interaction engineering, the above formula can be simplified to: e =K d (x d -x),

[0059] The robot's expected motion position x d Designed for human movement h , the expected stiffness matrix K d Designed for human body stiffness K h ,

[0060] The simplified impedance model is written in three-dimensional space as: Rewrite the above estimate as: Human body motion intention and human body stiffness parameter estimation

[0061] Build a quadratic cost function to evaluate accuracy: According to the human body dynamics model, Can be regarded as K h , assuming Obeys Gaussian distribution, so K h The random variable set κ obeys the following distribution: κ~N(μ,σ 2 ), N(*) is the Gaussian distribution function.

[0062] Considering the actual stiffness parameter K of the human body h1 It can be regarded as μ.2 In this case, μ can be estimated.

[0063] is the estimated value of μ.

[0064] The predicted probability distribution of the stiffness parameter p(μ) is: From the Bayesian estimation, we can know that p(μ|κ) is as follows: p(κ|μ) is the joint probability distribution, The updated probability distribution p(μ|κ) can be obtained as: p(μ|κ)=αp(κ|μ)p(μ), where α is used to absorb irrelevant terms.

[0065] The above formulas all obey Gaussian distribution.

[0066] in

[0067]

[0068] Human body stiffness parameters The estimated value remains at arrive Within the range, that is Human motion intention estimation x1 represents the one-dimensional position.

[0069] Obeying Gaussian distribution, the corresponding human motion intention estimation Also follows a Gaussian distribution:

[0070] As the estimation accuracy increases, μ x Converge to σ x Converges to zero.

[0071] Algorithm 2: Optimization of the motion trajectory of the constrained robot

[0072] The expected admittance model in the task space is: As an intermediate variable, the admittance model is decomposed into various dimensions of the task space: The reference trajectory is obtained by the soft saturation function in η(0η<1) is a constant very close to 1, satisfying: in For constraints.

[0073] The soft saturation function ensures that It is twice differentiable and constrained in the task space, ensuring that the subjective movement intention of the robot does not exceed the constraint boundary and preliminarily realizes the constraints in path planning.

[0074] On the basis of realizing the preliminary constraints, the IBLF method is used to constrain the system output.

[0075] x1=x,

[0076] Define the error variables z1, z2 as: z1 = x1-x r , z2=x2-α, α=[α1,α2,...,α n ] T is a dummy control variable.

[0077] One of the control objectives is to keep the system output x1 within the constraint region, that is, In order to be within the constraints, the following IBLF is designed:

[0078]

[0079] The time derivative of V1 is:

[0080]

[0081] Dummy control variable α i The design is as follows: where k i ,i=1,2,...,n is a normal number.

[0082]

[0083] Designed for: The time derivative of V2 is:

[0084]

[0085] The control input f = f m Designed for:

[0086] We can get: The positive gain matrix K2>0, so z1,z2 will converge to zero, and x1 can remain within the constraint region. Algorithm 3: Use neural networks to estimate the uncertainty in the dynamic model.

[0087] Using RBFNN to approximate the uncertainty problem of the robot arm can improve the robot arm system's accuracy to x r Tracking accuracy:

[0088]

[0089] in, is the estimated weight of RBFNN, S(Z) is the basis function, is the input variable, and the adaptive update law is designed as: Γ i is a positive definite symmetric matrix, is a small normal number.

[0090] To estimate W *T S(Z): Where ε is the approximation error, W * is the optimal weight of RBFNN.

[0091] Embodiment 2

[0092] In this embodiment, a leg-arm robot guiding method is disclosed, including:

[0093] Real-time observation of the blind person’s position, path planning and navigation;

[0094] According to the observed posture of the blind person and the environment information, the robot adjusts the torso position, locally plans the motion trajectory of the end of the traction arm, and safely tows the blind person to the desired relative posture. Based on the acquired 3D point cloud information and depth information, a comprehensive evaluation is performed to form a human-machine safe passage area to ensure that the robot can tow the blind person safely in complex dynamic scenes.

[0095] A person skilled in the art can understand that all or part of the processes in the above-mentioned implementation method can be completed through computer commands, and all computer programs are stored in a readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. The above embodiments only express several implementation methods of the present invention. For those skilled in the art, any changes or modifications made without departing from the concept of the present invention are within the scope of protection of the present invention. Therefore, the scope of protection of the patent of the present invention shall be subject to the attached claims.

Claims

1. A traction-type blind-guiding leg-arm robot, characterized in that: include: A traction-type guiding leg-arm robot body, wherein the traction-type guiding leg-arm robot body is provided with a traction-type guiding leg-arm robot control module; The legs and feet of the traction-type guiding leg-arm robot are used to support the robot body; The mechanical arm, the traction-type guide leg-arm robot body is connected to the mechanical arm; the mechanical arm includes a traction arm, and the traction arm is provided with a flexible joint; A handle is connected to the mechanical arm; the traction-type blind-guiding leg-arm robot guides the blind through the handle on the mechanical arm; Visual servo traction control framework for posture adjustment: including impedance control module, impedance parameter optimization module, Bayesian human motion intention estimation module, position control module, and traction control module; first, the Bayesian human motion intention estimation module estimates the human motion intention and human impedance parameters, and transmits the information to the impedance control module and the impedance parameter optimization module. After the impedance parameters are learned, they are transmitted back to the Bayesian human motion intention estimation module. The impedance control module is connected with the position control module and the traction control module to realize the constraint of the subjective movement intention of the robot arm and to traction the blind with appropriate force; The visual servo traction control design for the posture adjustment of the blind is as follows: first, the quadruped robot is used as a stable operating platform for the traction arm, the robot leg configuration is determined according to the perceived environment, and the robot torso position is adjusted; secondly, the mapping relationship between the image space and the joint space of the traction arm is established, and the depth information is decoupled based on the depth independent interaction matrix, and the motion trajectory of the end of the traction arm is locally planned according to the image information; finally, for the traction arm dynamics model and the human-machine force interaction model, the impedance parameters are learned based on Bayesian estimation, and the impedance control strategy of the human in the loop is designed to safely traction the blind to the desired relative posture; The traction-type guiding leg-arm robot adopts a human-machine coordinated motion control method. First, a tipping moment optimizer is designed to determine the desired trunk posture according to the human-machine interaction force measurement value, the human-machine relative posture and environmental information; Secondly, based on the dynamic model of the single rigid body of the trunk and inverse kinematics, the desired trajectory of the trunk is converted into the output of five virtual controllers. After selecting the gait from the gait library, the terminal motion trajectories of the traction arm, swing foot and supporting foot are planned. Finally, according to the outputs of the five virtual actuators, for the dynamic model of a single leg / arm, the closed-chain leg-arm terminal position constraints, the traction arm and blind manpower constraints, the swing leg and supporting leg terminal motion constraints, the foot end friction cone constraints, the traction space constraints and the joint limit constraints are considered as a whole. A multi-objective optimization method based on differential dynamic programming and alternating directions of multipliers is proposed. The dynamic models of the legs / arms are solved in parallel to generate joint torques that meet all constraints, thereby realizing robot compliant traction and blind operation.

2. The traction-type blind-guiding leg-arm robot according to claim 1, characterized in that: The traction-type guide robot is movably connected to the robotic arm through a floating base; the rotational freedom is retained between the robotic arm and the traction-type guide leg-arm robot body, and there is a certain angle limit to adapt to the changes in the posture between the robot and the blind in complex outdoor environments.

3. The traction-type blind-guiding leg-arm robot according to claim 1, characterized in that: The control module of the traction-type guide leg-arm robot includes a rigid-flexible coupling traction arm and control module, a composite scene perception module, a blind person posture tracking module, and a tactile prompt module; the rigid-flexible coupling traction arm and control module is connected to the traction-type guide leg-arm robot body and is located in the middle position of the traction-type guide leg-arm robot; the composite scene perception module is connected to the traction-type guide leg-arm robot body and is located at the head position of the traction-type guide leg-arm robot; the blind person posture tracking module is connected to the traction-type guide leg-arm robot body and is located at the tail position of the traction-type guide leg-arm robot, and the tactile prompt module is located at the traction arm handle.

4. The traction-type blind-guiding leg-arm robot according to claim 3, characterized in that: The tactile prompt module is arranged on the handle.

5. The traction-type blind-guiding leg-arm robot according to claim 3, characterized in that: The composite scene perception module includes a laser radar, an RGB-D camera, a side camera and a force sensor.

6. The traction-type blind-guiding leg-arm robot according to claim 3, characterized in that: The blind posture tracking module includes a mobile gimbal equipped with a depth camera at the rear of the robot; The mobile gimbal observes the position of the blind person in real time; first, the traction-type guide leg-arm robot determines the position of the robot's legs and adjusts the position of the robot body according to the perceived environment; secondly, a mapping relationship between the image space and the joint space of the traction arm is established, and depth information decoupling is achieved based on the depth independent interaction matrix, and the motion trajectory of the end of the traction arm is locally planned according to the image information; finally, based on the traction arm dynamics model and the human-machine force interaction model, the impedance parameters are learned based on Bayesian estimation, and the impedance control strategy of the blind person in the loop is designed to safely traction the blind person to the desired relative position; The blind person follows the robot under the traction of the robotic arm. Based on the Bayesian estimation of human motion intention, the robotic arm actively follows the blind person by estimating the human motion intention and the traction force of the robotic arm. The mechanical arm is a rigid-flexibly coupled traction arm; The surrounding environment information is obtained through lidar and depth camera, and a lightweight separable convolutional integrated semantic segmentation network is constructed to integrate semantic information and deep features to realize a comprehensive evaluation method for human-machine safe passage and the generation of walkable areas based on blind path detection.

Images