A method and system for human-machine multi-source perception and collaborative control of a walking-assisting robot

Abstract

This invention discloses a multi-source human-machine perception and collaborative control method and system for an elderly-assisting walking companion robot. Wearable inertial sensors are used to collect the user's lower limb motion information, and the user's walking speed is predicted using an LSTM-based walking speed prediction algorithm. To eliminate the difference between the robot's actual and human speeds, the user's tactile force applied to the robot's armrests is collected to generate a compensation speed. Simultaneously, an RGB-D camera captures color and depth images of the environment, and an obstacle cost map is obtained using an environmental obstacle perception algorithm. Finally, based on an improved dynamic window algorithm, the collaboratively controlled robot speed is output, decomposed into target speeds for the left and right wheels using a two-wheel differential inverse kinematics model, and the motor speed is regulated using PID control. This method can solve the problem of elderly people improperly manipulating robots during obstacle avoidance.

Description

Technical Field

[0001] The present invention belongs to the field of human-computer interaction and robot control technology, and specifically relates to a human-computer multi-source perception and collaborative control method and system for an elderly-assisting walking companion robot. Background Art

[0002] Elderly walking companion robots are designed for elderly individuals with some walking ability but weak lower limbs. The robot closely accompanies the elderly during walking, its frame providing physical support and protection. Currently, the control of elderly walking companion robots relies primarily on recognizing the user's walking intentions, lacking their own perception and judgment of environmental obstacles. However, elderly individuals have decreased reaction speeds and poor motor coordination, making them prone to collisions due to improper operation and inattention in narrow, obstructive roads. Therefore, controlling the robot to avoid obstacles, guided by the user's walking intentions, has become a key challenge in the development of elderly walking companion robots.

[0003] Therefore, there is an urgent need to develop a collaborative control method and system for elderly walking companion robots, which can accurately identify the walking intentions of the elderly and perform fine motion planning in combination with environmental obstacle information, thereby improving the human-computer interaction friendliness and safety of the elderly walking companion robots, and realizing the application and promotion of elderly walking companion robots. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies in the above-mentioned existing technologies and provide a method and system for human-machine multi-source perception and collaborative control of an elderly-assisting walking robot, so as to solve the technical problem of improper manipulation of the robot by the elderly during obstacle avoidance.

[0005] The present invention adopts the following technical solutions:

[0006] A method for human-machine multi-source perception and collaborative control of a walking companion robot includes the following steps:

[0007] The intended walking speed is determined based on the predicted walking speed and the compensation speed; the environmental obstacle cost map is obtained through the obstacle perception and positioning algorithm; the intended walking speed and the environmental obstacle cost map are combined to plan a safe movement path through an improved dynamic window algorithm, and the robot movement speed corresponding to the movement path is converted into control commands for the robot's left and right drive wheel motors through a two-wheel differential drive inverse kinematics model. The PWM values ​​of the left and right drive wheel motors are adjusted in real time based on the actual wheel speed of the robot to achieve collaborative control.

[0008] Specifically, the IMU signal from the outside of the calf is collected, and the walking speed is obtained through a walking speed prediction algorithm based on a long short-term memory neural network. The tactile force signal from the robot armrest is collected and a compensation speed is generated, which is added to the predicted walking speed as the walking intention speed.

[0009] Furthermore, the walking speed prediction algorithm is specifically as follows:

[0010] The feature vector is constructed based on the three-axis acceleration and angular velocity signals of the left and right calves at time i ; The feature vectors of the past M frames form a time series , as the input of the walking speed prediction algorithm; Perform the maximum and minimum normalization on a unified scale, then input the network model with the LSTM layer and the fully connected layer stacked, and output the predicted walking speed at time i+N .

[0011] Furthermore, walking speed Specifically:

[0012]

[0013] in, is the predicted walking linear speed of the user at the i+Nth moment, is the predicted walking angular velocity of the user at the i+Nth moment.

[0014] Furthermore, the walking intention speed is as follows:

[0015] Collect the tactile force signals applied by the user to the robot armrest and calculate the resultant force vectors of the left and right hands ; Under the discrete control system, the tactile force is integrated to obtain the compensation speed V comp Vector; the sum of the compensation speed and the predicted speed is the walking intention speed .

[0016] Furthermore, the compensation speed V comp Vector for:

[0017]

[0018] in, is the control period of the system, is the speed feedback gain, For tactile force, To compensate for speed.

[0019] Specifically, obtaining the environmental obstacle cost map is as follows:

[0020] The system obtains color images from an RGB-D camera and uses a two-dimensional object detection algorithm to output the category and bounding box of the obstacle. At the same time, it obtains a depth image from the RGB-D camera and extracts the view frustum point cloud of the obstacle bounding box. The view frustum point cloud is preprocessed with voxel downsampling, depth-direction pass-through filtering, and Euclidean clustering denoising. The preprocessed point cloud is projected onto a two-dimensional grid map to generate an environmental obstacle cost map.

[0021] Specifically, the safe movement path is planned by improving the dynamic window algorithm as follows:

[0022] Update the current status of the elderly companion robot , a speed sampling space is generated based on the kinematic constraints of the robot; sampling is performed in the speed sampling space to generate a set of sampling velocities; forward simulation is performed at the sampling velocity for a period of time to generate a set of sampling trajectories; the optimal trajectory is selected through an improved evaluation function; the velocity vector corresponding to the optimal trajectory is converted into the speed of the left and right driving wheel motors through the two-wheel differential drive inverse kinematics model, the actual wheel speed of the robot is measured, and the PWM value of the motor is adjusted through the PID controller.

[0023] Further, improved evaluation function Specifically:

[0024]

[0025] in, is the sampling speed of the i-th group, is the weight coefficient of the three evaluation sub-functions, σ( ) is the normalization processing of each sub-function, For the sake of Figure 1 Consistency evaluation subfunction, is the obstacle distance evaluation subfunction, It is the sub-function for evaluating speed smoothness.

[0026] In a second aspect, an embodiment of the present invention provides a multi-source human-machine perception and collaborative control system for an elderly-assisting walking companion robot, comprising:

[0027] A multi-source perception module, including two wearable IMUs, a tactile force sensor, and an RGB-D camera, is used to collect IMU signals, tactile force signals, and RGB-D images of the environment;

[0028] Collaborative control module, including upper computer, lower computer, wheel hub motor driver and wheel hub motor;

[0029] The upper computer obtains the IMU signal and RGB-D image, and the lower computer obtains the tactile force signal and transmits it to the upper computer; the RGB-D image is calculated by the upper computer and outputs the control speed of the hub motor to the lower computer. The lower computer translates it into PWM value through PID and controls the hub motor through the hub motor driver.

[0030] Compared with the prior art, the present invention has at least the following beneficial effects:

[0031] A multi-source human-machine perception and collaborative control method for a walking companion robot for the elderly not only enables the robot to accurately identify the user's walking intention and closely accompany the user's movement, but also can identify and locate obstacles when the user improperly controls the robot or fails to notice them, assisting the user in avoiding obstacles and preventing collisions between the user and the robot, thereby improving the friendliness and safety of the control of the walking companion robot.

[0032] Furthermore, when using tactile force or IMU information to estimate the user's walking intention, there are problems of low estimation accuracy and lack of adaptability. The present invention collects the user's IMU signal and uses a walking speed prediction algorithm based on deep learning to infer the walking speed. It can fit the walking speed characteristics of different users and has the advantages of high accuracy and high versatility; collecting the tactile force generated by the relative speed between man and machine can further eliminate the error in estimating walking speed.

[0033] Furthermore, because walking is a cyclical motion, a deep learning-based walking speed prediction algorithm uses a dataset to learn the relationship between the three-axis acceleration and angular velocity signals of the lower limbs during walking and walking speed. This algorithm can predict walking speed for the next N frames based on the signals from the past M frames, reducing communication and control delays while also providing a certain degree of versatility in predicting walking speed for different users.

[0034] Furthermore, when the movement speed planned by the robot for obstacle avoidance deviates significantly from the user's actual walking speed, in order to protect the user from being pushed down by the robot, a compensation speed based on tactile force is set to adjust the predicted speed. The sum of the compensation speed and the predicted walking speed is the user's intended walking speed. Using this speed as the human input in collaborative control can effectively reduce the difference between the actual moving speeds of humans and machines.

[0035] Furthermore, through color Figure 2 The method of performing 3D target detection and then extracting the view cone in the depth map to obtain the point cloud of the obstacle and generate the obstacle cost map. Compared with the method of directly performing 3D target detection and instance segmentation on the depth map point cloud, it can greatly reduce the calculation of point cloud data, thereby improving the speed of obstacle perception.

[0036] Furthermore, the improved dynamic window algorithm incorporates human walking intentions into the robot path planning algorithm by setting an improved evaluation function, and then selects a safe moving path that meets the user's walking intentions and does not encounter obstacles based on the score. It combines human decision-making ability with the robot's precise planning ability, thereby achieving better path planning.

[0037] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0038] In summary, the present invention addresses the problems of decreased reaction speed and poor motor coordination among the elderly, making it difficult for them to control robots to avoid obstacles accurately while walking. It improves the accuracy of the robot's perception of the user's walking intention and the robot's ability to identify obstacles in outdoor walking scenes. Based on the results of multi-source perception, collaborative control is used to combine the intelligence of humans and robots for decision-making, thereby improving the friendliness and safety of the control of the elderly walking companion robot and ensuring the walking safety of the elderly.

[0039] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart of the present invention;

[0041] Figure 2 This is a schematic diagram of the human-machine multi-source perception and collaborative control system of the elderly-assisting walking robot of the present invention.

[0042] Among them, 1. Tactile force sensor; 2. RGB-D camera; 3. Wearable IMU; 4. Front wheel; 5. Rear wheel; 6. Chassis. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] In the description of the present invention, it is to be understood that the terms “include” and “comprise” indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0045] It should also be understood that the terms used in the present specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0046] It should be further understood that the term "and / or" as used in the present specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0047] It should be understood that although the terms "first," "second," and "third" may be used to describe preset ranges in embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are merely used to distinguish one preset range from another. For example, without departing from the scope of embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0048] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0049] The accompanying drawings illustrate various schematic diagrams of structures according to embodiments disclosed herein. These figures are not drawn to scale; for clarity, some details are exaggerated and some details may be omitted. The shapes of the various regions and layers shown in the figures, as well as their relative sizes and positional relationships, are merely exemplary and may deviate in practice due to manufacturing tolerances or technical limitations. Those skilled in the art may design regions / layers with different shapes, sizes, and relative positions as needed.

[0050] The present invention provides a human-machine multi-source perception and collaborative control method for a walking companion robot for the elderly. A wearable inertial sensor is used to collect the user's lower limb motion information, and the user's walking speed is predicted by a walking speed prediction algorithm based on LSTM. In order to eliminate the difference between the actual movement speed of the robot and the actual movement speed of the person, the tactile force applied by the user on the robot's armrest is collected to generate a compensation speed. At the same time, color and depth images of the environment are collected by an RGB-D camera, and an obstacle cost map is obtained by an environmental obstacle perception algorithm. Finally, based on an improved dynamic window algorithm, the collaboratively controlled robot speed is output, which is decomposed into the target speeds of the left and right wheels through a two-wheel differential inverse kinematics model, and the motor speed is regulated by PID control.

[0051] The present invention provides a human-machine multi-source perception and collaborative control method for an elderly-assisting walking companion robot, comprising the following steps:

[0052] S1. Collect the signal of the IMU worn on the outside of the user's left and right calves, and obtain the predicted walking speed of the user through the walking speed prediction algorithm ;

[0053] The walking speed prediction algorithm is based on the long short-term memory neural network (LSTM). The specific steps are:

[0054] S101: Collect the three-axis acceleration and angular velocity signals of the user's left and right calves at time i and form a feature vector ;

[0055] Eigenvector :

[0056]

[0057] in, represents acceleration, represents angular velocity, the subscript represents the left and right legs, and the superscript represents the time.

[0058] S102: Construct a time series from the feature vectors of the past M frames , as input to the walking speed prediction algorithm;

[0059] Time Series for:

[0060]

[0061] in, is the angular velocity characteristic vector of the left and right calves at the past i moment, is the length of the historical time window.

[0062] S103, yes The maximum and minimum values ​​are normalized to a unified scale, and then input into the network model with the LSTM layer and the fully connected layer stacked, and the predicted walking speed at time i+N is output.

[0063] Walking speed at time i+N for:

[0064]

[0065] in, is the predicted walking linear speed of the user at the i+Nth moment, is the predicted walking angular velocity of the user at the i+Nth moment.

[0066] S2. Collect the tactile force signal applied by the user to the robot armrest and generate the compensation speed. The sum of the compensation speed and the predicted walking speed is the walking intention speed. ;

[0067] S201: Collect the tactile force signal applied by the user to the robot armrest and calculate the left and right hand resultant force vectors ;

[0068] Left and right hand resultant force vector 、 for:

[0069]

[0070] in, For the left hand tactile sense, is the tactile sense of the right hand, is the distance between the left and right tactile force sensors.

[0071] S202: Integrate the tactile force to obtain a compensation speed V under a discrete control system. comp vector;

[0072] Compensation speed V comp Vector for:

[0073]

[0074] in, is the control period of the system, is the speed feedback gain, For tactile force, To compensate for speed.

[0075] S203: The sum of the compensation speed and the predicted speed is the walking intention speed .

[0076] S3, collects images from the RGB-D camera fixed under the robot's armrest and obtains the environmental obstacle cost map through the obstacle perception and positioning algorithm;

[0077] S301, obtaining a color image from an RGB-D camera and outputting the category and bounding box of the obstacle using a two-dimensional object detection algorithm;

[0078] S302, obtaining a depth image of the RGB-D camera at the same time, and extracting a frustum point cloud of the obstacle bounding box;

[0079] S303, performing pre-processing operations of voxel downsampling, depth-direction straight-through filtering, and Euclidean clustering denoising on the view frustum point cloud;

[0080] S304: Project the processed point cloud onto a two-dimensional grid map to generate a cost map of obstacles.

[0081] S4. A safe movement path is planned by combining the user's intended walking speed with the obstacle cost map through an improved dynamic window algorithm. The output of the planned speed is converted into control commands for the robot's left and right drive wheel motors through a two-wheel differential drive inverse kinematics model.

[0082] S401, update the current status of the elderly companion robot ,Generate the velocity sampling space based on the kinematic constraints of the robot;

[0083] S402, sampling in the speed sampling space to generate a sampling speed set;

[0084] S403, forward simulation for a period of time at the sampling speed to generate a set of sampling trajectories;

[0085] S404, selecting the optimal trajectory through the improved evaluation function;

[0086] The improved evaluation function is the weighted sum of three evaluation sub-functions:

[0087]

[0088] in, is the sampling speed of the i-th group, is the weight coefficient of the three evaluation sub-functions, σ( ) is the normalization processing of each sub-function.

[0089] meaning Figure 1 The calculation method of the consistency evaluation subfunction is:

[0090]

[0091] in, The user's walking speed.

[0092] The calculation method of the obstacle distance evaluation sub-function is:

[0093]

[0094] in, is the coordinate position at time j on the i-th trajectory, cell() - the cost value corresponding to this point on the cost map.

[0095] The calculation method of the speed smoothness evaluation subfunction is:

[0096]

[0097] in, is the current linear velocity and angular velocity of the robot.

[0098] S405; The velocity vector corresponding to the optimal trajectory is converted into the left and right drive wheel motor speeds through the two-wheel differential drive inverse kinematics model, the actual wheel speed of the robot is measured, and the PWM value of the motor is adjusted through the PID controller.

[0099] In another embodiment of the present invention, a human-machine multi-source perception and collaborative control system for a walking companion robot for the elderly is provided. The system can be used to implement the above-mentioned human-machine multi-source perception and collaborative control method for the walking companion robot for the elderly. Specifically, the human-machine multi-source perception and collaborative control system for the walking companion robot for the elderly includes a multi-source perception module and a collaborative control module.

[0100] The multi-source perception module includes two wearable IMUs (inertial measurement units) (IMUs), a tactile force sensor (1) on the robot's armrest, and an RGB-D camera (2). This module is used to collect IMU signals from the user's calf, tactile force signals for human-machine interaction, and RGB-D images of the environment.

[0101] Collaborative control module, including upper computer, lower computer, wheel hub motor driver and wheel hub motor;

[0102] The host computer is responsible for communicating with various sensors to estimate walking speed and perceive environmental obstacles. It also performs coordinated control based on the results of multi-source perception and outputs the control speeds of the left and right motors.

[0103] The lower computer is responsible for the PID control of the hub motor and outputs PWM values ​​to achieve speed regulation of the hub motor through the hub motor driver.

[0104] See also Figure 2 When the elderly-assisting walking companion robot is in use, multi-source information collection is performed through a multi-source perception module, including a tactile force sensor 1 on the robot's armrest, an RGB-D camera 2, and wearable IMUs 3 on the outside of the left and right calves, which are used to collect IMU signals from the user's calves, tactile force signals for human-computer interaction, and RGB-D images of the environment; the motion control of the elderly-assisting walking companion robot is completed through a collaborative control module, and the upper computer, lower computer, and hub motor driver are integrated in the chassis 6. The front wheel 4 of the elderly-assisting walking companion robot is a controlled hub motor, and the rear wheel 5 is a supporting caster.

[0105] The upper computer is connected to the IMU via Bluetooth to collect IMU signals. The lower computer and the tactile force sensor collect tactile force signals through serial communication and transmit them to the upper computer through the serial port for unified calculation. The upper computer and the RGB-D camera collect image information through USB communication. After calculation, the upper computer outputs the control speed of the left and right motors and sends it to the lower computer through serial communication. The lower computer translates the PID into PWM values ​​through the motor driver to control the motor.

[0106] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to assist the operation of the multi-source perception and collaborative control method of the elderly companion robot, including:

[0107] The intended walking speed is determined based on the predicted walking speed and the compensation speed; the environmental obstacle cost map is obtained through the obstacle perception and positioning algorithm; the intended walking speed and the environmental obstacle cost map are combined to plan a safe movement path through an improved dynamic window algorithm, and the robot movement speed corresponding to the movement path is converted into control commands for the robot's left and right drive wheel motors through a two-wheel differential drive inverse kinematics model. The PWM values ​​of the left and right drive wheel motors are adjusted in real time based on the actual wheel speed of the robot to achieve collaborative control.

[0108] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a terminal device, used to store programs and data. It is understood that the computer-readable storage medium herein may include both built-in storage media in the terminal device and, of course, extended storage media supported by the terminal device. The computer-readable storage medium provides storage space, which stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for being loaded and executed by a processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device.

[0109] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the method for multi-source human-machine perception and collaborative control of the elderly companion walking robot in the above embodiment; the processor may load and execute the following steps:

[0110] The intended walking speed is determined based on the predicted walking speed and the compensation speed; the environmental obstacle cost map is obtained through the obstacle perception and positioning algorithm; the intended walking speed and the environmental obstacle cost map are combined to plan a safe movement path through an improved dynamic window algorithm, and the robot movement speed corresponding to the movement path is converted into control commands for the robot's left and right drive wheel motors through a two-wheel differential drive inverse kinematics model. The PWM values ​​of the left and right drive wheel motors are adjusted in real time based on the actual wheel speed of the robot to achieve collaborative control.

[0111] In summary, the present invention provides a multi-source human-machine perception and collaborative control method and system for a walking companion robot. This method utilizes a wearable IMU sensor to acquire the user's calf motion information and accurately predicts the user's walking speed based on an LSTM network. A tactile force sensor is used to adjust the predicted walking speed to eliminate the difference between the actual human and machine motion speeds, thus avoiding collisions. An RGB-D camera is used to rapidly identify and locate obstacles and generate an obstacle cost map. When an elderly person encounters an obstacle while using the walking companion robot, the system combines the user's global decision-making ability with the robot's precise planning and control capabilities to assist the user in safely circumventing the obstacle. Compared to walking companion robots that rely entirely on user control, this method has the advantages of reducing the tactile force required to control the robot, improving walking efficiency, and providing increased safety.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0113] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0114] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0115] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0118] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0122] The above content is only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for multi-source perception and collaborative control of a walking companion robot, characterized in that: The following steps are involved: The intended walking speed is determined based on the predicted walking speed and the compensation speed; the environmental obstacle cost map is obtained through the obstacle perception and positioning algorithm; the walking intention speed and the environmental obstacle cost map are combined to plan a safe movement path through the improved dynamic window algorithm. The robot movement speed corresponding to the movement path is converted into the control command of the robot's left and right drive wheel motors through the two-wheel differential drive inverse kinematics model. The PWM values ​​of the left and right drive wheel motors are adjusted in real time based on the actual wheel speed of the robot to achieve coordinated control. The safe movement path planned by the improved dynamic window algorithm is as follows: Update the current status of the elderly companion robot , based on the kinematic constraints of the robot, a velocity sampling space is generated; sampling is performed in the velocity sampling space to generate a sampling velocity set; Simulate forward for a period of time at the sampling speed to generate a set of sampling trajectories; The optimal trajectory is selected through the improved evaluation function; the velocity vector corresponding to the optimal trajectory is converted into the left and right drive wheel motor speeds through the two-wheel differential drive inverse kinematics model, the actual wheel speed of the robot is measured, and the PWM value of the motor is adjusted through the PID controller. The improved evaluation function Specifically: in, is the sampling speed of the i-th group, is the weight coefficient of the three evaluation sub-functions, σ( ) is the normalization processing of each sub-function, is the intention consistency evaluation subfunction, is the obstacle distance evaluation subfunction, It is the speed smoothness evaluation subfunction; The calculation method of the intention consistency evaluation sub-function is: in, The user's walking speed; The calculation method of the obstacle distance evaluation sub-function is: in, is the coordinate position of the i-th trajectory at time j, cell() - the cost value corresponding to the point on the cost map; The calculation method of the speed smoothness evaluation subfunction is: in, is the current linear velocity and angular velocity of the robot.

2. The method for multi-source human-machine perception and collaborative control of a walking companion robot for the elderly according to claim 1 is characterized in that: The IMU signal from the outside of the calf is collected, and the walking speed is obtained through a walking speed prediction algorithm based on a long short-term memory neural network. The tactile force signal from the robot armrest is collected and a compensation speed is generated, which is added to the predicted walking speed as the walking intention speed.

3. The method for multi-source human-machine perception and collaborative control of a walking companion robot for the elderly according to claim 2 is characterized in that: The walking speed prediction algorithm is as follows: The feature vector is constructed based on the three-axis acceleration and angular velocity signals of the left and right calves at time i ; The feature vectors of the past M frames form a time series , as the input of the walking speed prediction algorithm; Perform the maximum and minimum normalization on a unified scale, then input the network model with the LSTM layer and the fully connected layer stacked, and output the predicted walking speed at time i+N .

4. The method for multi-source human-machine perception and collaborative control of a walking companion robot for the elderly according to claim 3 is characterized in that: Walking speed Specifically: in, is the predicted walking linear speed of the user at the i+Nth moment, is the predicted walking angular velocity of the user at the i+Nth moment.

5. The method for multi-source human-machine perception and collaborative control of a walking companion robot for the elderly according to claim 2 is characterized in that: The walking intention speed is as follows: Collect the tactile force signals applied by the user to the robot armrest and calculate the resultant force vectors of the left and right hands ; Under the discrete control system, the tactile force is integrated to obtain the compensation speed V comp Vector; the sum of the compensation speed and the predicted speed is the walking intention speed .

6. The method for multi-source human-machine perception and collaborative control of a walking companion robot for the elderly according to claim 5 is characterized in that: Compensation speed V comp Vector for: in, is the control period of the system, is the speed feedback gain, For tactile force, To compensate for speed.

7. The method for multi-source human-machine perception and collaborative control of a walking companion robot for the elderly according to claim 1 is characterized in that: Obtaining the environmental obstacle cost map is as follows: The system obtains color images from an RGB-D camera and uses a two-dimensional object detection algorithm to output the category and bounding box of the obstacle. At the same time, it obtains a depth image from the RGB-D camera and extracts the view frustum point cloud of the obstacle bounding box. The view frustum point cloud is preprocessed with voxel downsampling, depth-direction pass-through filtering, and Euclidean clustering denoising. The preprocessed point cloud is projected onto a two-dimensional grid map to generate an environmental obstacle cost map.

8. A multi-source perception and collaborative control system for a walking companion robot, characterized in that: The method for human-machine multi-source perception and collaborative control of an elderly-assisting walking companion robot according to any one of claims 1 to 7 comprises: A multi-source perception module, including two wearable IMUs, a tactile force sensor, and an RGB-D camera, is used to collect IMU signals, tactile force signals, and RGB-D images of the environment; Collaborative control module, including upper computer, lower computer, wheel hub motor driver and wheel hub motor; The host computer obtains the IMU signal and RGB-D image, and the lower computer obtains the tactile force signal and transmits it to the host computer; the RGB-D image is calculated by the host computer and outputs the control speed of the hub motor to the lower computer. The lower computer translates it into PWM value through PID and controls the hub motor through the hub motor driver. The improved evaluation function Specifically: in, is the sampling speed of the i-th group, is the weight coefficient of the three evaluation sub-functions, σ( ) is the normalization processing of each sub-function, is the intention consistency evaluation subfunction, is the obstacle distance evaluation subfunction, It is the speed smoothness evaluation subfunction; The calculation method of the intention consistency evaluation sub-function is: in, The user's walking speed; The calculation method of the obstacle distance evaluation sub-function is: in, is the coordinate position of the i-th trajectory at time j, cell() - the cost value corresponding to the point on the cost map; The calculation method of the speed smoothness evaluation subfunction is: in, is the current linear velocity and angular velocity of the robot.

Images