Robot motion control method and device, electronic equipment and storage medium
By acquiring environmental information and joint parameters, the robot's motion mode is switched to adapt to the environment, solving the problem of low speed and poor stability of wheeled mobile robots in human-inhabited environments, and realizing stable movement of the robot in complex environments.
Patent Information
- Application Number
- CN202310621872.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-29
- Publication Date
- 2024-11-29
AI Technical Summary
Existing wheeled mobile robots have low speed and poor stability in real human living environments, and lack effective motion control methods to improve their environmental adaptability.
By acquiring the robot's current environmental information and joint motion parameters, the environment type is determined and the robot is switched to an adaptive target motion mode, and the robot's joint motion parameters are configured to adapt to different environments.
This enhances the robot's environmental adaptability, enabling it to move stably in different environments and adapt to various application scenarios.
Smart Images

Figure CN119036433A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to computer technology, and in particular to a robot motion control method, device, electronic device and computer-readable storage medium. Background Art
[0002] With the development of robot technology, various robots are increasingly used in the fields of smart factories, smart logistics, smart warehousing, etc. to significantly improve the degree of automation and reduce human resource costs. Common mobile robots include legged robots and wheeled robots.
[0003] In order to enable robots to move flexibly and efficiently and adapt to complex terrain environments, wheeled mobile robots are usually used. The low speed and poor stability of wheeled mobile robots limit their application in actual human settlement environments.
[0004] Related technologies There is currently no better robot motion control method that can improve the robot's ability to adapt to the environment. Summary of the invention
[0005] The embodiments of the present application provide a robot motion control method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the robot's environmental adaptability.
[0006] The technical solution of the embodiment of the present application is implemented as follows:
[0007] The present application provides a robot motion control method, the method comprising:
[0008] Acquire environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot;
[0009] Determine the environment type of the current environment based on the environment information;
[0010] Determining a current posture of the robot based on motion parameters of the joints;
[0011] Determine the position information of the robot in the current environment based on the environmental information and the motion parameters of the joints;
[0012] In response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type;
[0013] The motion parameters of the joints of the robot are configured based on the target motion mode corresponding to the environment type, wherein the motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
[0014] The present application provides a robot motion control device, comprising:
[0015] A sensor module configured to obtain environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot;
[0016] A state estimation module, configured to determine an environment type of the current environment based on the environment information;
[0017] The state estimation module is configured to determine the current posture of the robot based on the motion parameters of the joints;
[0018] The state estimation module is configured to determine the position information of the robot in the current environment based on the environmental information and the motion parameters of the joints;
[0019] a motion control module, configured to switch the current motion mode of the robot to a target motion mode corresponding to the environment type in response to the current posture and the position information satisfying a motion mode switching condition;
[0020] The motor control module is configured to configure motion parameters of the joints of the robot based on the target motion mode corresponding to the environment type, wherein the motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
[0021] An embodiment of the present application provides an electronic device, the electronic device comprising:
[0022] A memory for storing computer executable instructions;
[0023] The processor is used to implement the robot motion control method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.
[0024] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the robot motion control method provided in the embodiment of the present application when executed by a processor.
[0025] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the robot motion control method provided by the embodiment of the present application is implemented.
[0026] The embodiments of the present application have the following beneficial effects:
[0027] By acquiring environmental information and the current posture of the robot, the target motion mode that adapts to the current environment is determined, and the robot's motion parameters are configured based on the parameters of the target motion mode, so that the robot's motion mode is switched to a motion mode that adapts to the environment, thereby improving the robot's environmental adaptability, enabling the robot to move stably in different types of environments, and enabling the robot to adapt to different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 It is a schematic diagram of an application mode of the motion control method of a robot provided in an embodiment of the present application;
[0029] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0030] FIG. 3A to FIG. 3D is a flow chart of a robot motion control method provided in an embodiment of the present application;
[0031] 4A to 4D is a schematic diagram of the robot structure provided in an embodiment of the present application;
[0032] FIG. 5A to FIG. 5B is a schematic diagram of the robot movement provided by the embodiment of the present application;
[0033] Fig. 6A It is a schematic diagram of the principle of the motion control method of the robot provided in the embodiment of the present application;
[0034] Figure 6B It is a module schematic diagram of a state control machine provided in an embodiment of the present application;
[0035] Figure 6C It is a schematic diagram of the modules provided in the embodiment of the present application. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0037] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0038] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0039] It should be pointed out that the collection and processing of relevant data in this application (for example, environmental information collected when the robot is running) should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.
[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0041] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.
[0042] 1) Mobile robots, such as legged robots and wheeled robots. In order to enable robots to move flexibly and efficiently and adapt to complex terrain environments, wheeled and legged mobile robots are usually used. Wheeled and legged mobile robots are based on legged robots, for example, wheels are added to the bottom of their foot mechanisms, so that wheeled and legged mobile robots have both the agility of wheeled movement and the flexibility of legged movement, and can adapt to a variety of complex terrains.
[0043] 2) Multimodal mobile robots: The motion mode of a multimodal mobile robot can be switched between foot-based and wheel-based. For example, a multimodal mobile robot switches from dual wheels to dual feet.
[0044] 3) Convolutional Neural Networks (CNN) is a type of feed forward neural network (FNN) that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have the ability of representation learning and can perform shift-invariant classification on input images according to their hierarchical structure.
[0045] 4) Inertial Measurement Unit (IMU), a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object. The inertial sensor is mainly composed of three accelerometers, an acceleration sensor, three gyroscopes and a solution circuit. The accelerometer detects the acceleration signal of the object in the three independent axes of the carrier coordinate system, while the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system, measures the angular velocity and acceleration of the object in three-dimensional space, and uses this to solve the object's attitude.
[0046] 5) Distance sensor, also called displacement sensor, is a type of sensor used to sense the distance between it and an object to complete a preset function. Distance sensors can be divided into optical distance sensors, infrared distance sensors, ultrasonic distance sensors and other types according to their working principles.
[0047] 6) Tactile sensor: Tactile sensor is a sensor used in robots to imitate the sense of touch. According to the function, it can be divided into contact sensor, torque sensor, pressure sensor and slip sensor.
[0048] The embodiments of the present application provide a robot motion control method, a robot motion control device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the robot's environmental adaptability.
[0049] The following describes an exemplary application of the electronic device provided by the embodiment of the present application. The electronic device provided by the embodiment of the present application can be implemented as a laptop, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), a vehicle terminal, a virtual reality (VR) device, an augmented reality (AR) device, and other types of user terminals), and can also be implemented as a server. Below, an exemplary application when the device is implemented as a terminal device will be described.
[0050] refer to Figure 1 , Figure 1 is a schematic diagram of an application mode of the robot motion control method provided in an embodiment of the present application; for example, Figure 1 The server 200, the robot body 100, the network 300 and the terminal device 400 are involved. The terminal device 400 is connected to the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.
[0051] For example, the mechanical structure of the robot body 100 is controlled by the terminal device 400. The terminal device 400 calls the robot motion control method provided in the embodiment of the present application, analyzes the environmental information and motion parameters collected by various sensors provided on the robot, determines the corresponding motion mode of the robot in the current environment, and controls the robot to switch to the corresponding motion mode, so that the robot can move in a motion mode that adapts to the current environment.
[0052] The embodiments of the present application can be implemented through database technology. In short, a database can be regarded as an electronic file cabinet where electronic files are stored. Users can add, query, update, delete, etc. data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.
[0053] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as Structured Query Language (SQL), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages at the same time.
[0054] The embodiments of the present application can also be implemented through cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application. It can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, as well as the promotion of search services, social networks, mobile commerce and open collaboration, each item may have its own hash code identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.
[0055] In some embodiments, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The electronic device may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal device and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present invention.
[0056] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The electronic device shown may be a terminal device, and the terminal device 400 includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .
[0057] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.
[0058] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.
[0059] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.
[0060] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.
[0061] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.
[0062] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;
[0063] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;
[0064] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);
[0065] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.
[0066] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The robot motion control device 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a sensing module 4551, a state estimation module 4552, a motion control module 4553, and a motor control module 4554. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.
[0067] The motion control method of the robot provided in the embodiment of the present application will be explained in combination with the exemplary application and implementation of the electronic device provided in the embodiment of the present application.
[0068] The following describes the robot motion control method provided by the embodiment of the present application. As mentioned above, the electronic device that implements the robot motion control method of the embodiment of the present application can be a terminal device, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.
[0069] See also Figure 3A , Figure 3A is a flow chart of the robot motion control method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.
[0070] In step 301, environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot are obtained.
[0071] The robot in the embodiment of the present application has different motion modes and can switch between different motion modes. For the convenience of explanation, the following is an explanation of a type of robot controlled by the motion control method of the robot in the embodiment of the present application in conjunction with the accompanying drawings. Figure 4A , Figure 4A 401, a body part 402, a waist part 403, a leg part 406 (including an inner leg 404 and an outer leg 405), a wheel 407 at the end of each leg, an upper limb 408, a pitch rotation center 409, and a hip rotation center 410. The embodiment of the present application mainly describes the movement of the robot, that is, the movement of the part of the robot that contacts the ground is explained, and the upper limbs are not described in detail here. Figure 4B , Figure 4B 501 is a schematic diagram of the robot structure provided in the embodiment of the present application; the robot also includes a lateral swing rotation center 501, which is used to rotate the robot body around a direction perpendicular to the ground. It should be understood that there are many different types of multimodal mobile robots. The robot motion control method provided in the embodiment of the present application can be applied to different types of robots with motion mode switching capabilities, and is not limited to Figure 4A The robot used as an example.
[0072] For example, the motion parameters include: angular velocity, angle, acceleration, and moving speed of the motor joints of the robot. Step 301 can be implemented by calling a variety of sensors set on the robot. The sensors set on the robot include: positioning (GPS) sensor, foot tactile sensor, foot force and torque sensor, inertia sensor (Inertia Measurement Unit, IMU), distance sensor (TOF), and remote control and image processor (GUI) responsible for human-machine interaction input, camera (Camera) and visual sensor, etc.
[0073] In some embodiments, reference Figure 3B , Figure 3B is a flow chart of a robot motion control method provided in an embodiment of the present application, Figure 3A Step 301 in can be Figure 3B Steps 3011 to 3015 in are implemented as described in detail below.
[0074] In step 3011, sensors of the joints of the robot are called to obtain the angular velocity, angle and movement speed of the joints, and the angular velocity, angle and movement speed are used as movement parameters.
[0075] For example, the sensors of the robot's joints are inertial sensors, which are used to measure the three-axis attitude angle (or angular velocity) and acceleration of an object. The inertial sensors installed at the motor joints can obtain the angular velocity, angle and movement speed of the motor joints.
[0076] In step 3012, the distance sensor of the robot is called to obtain the distance between the surface of the robot and the obstacles in the current environment.
[0077] For example, the distance sensor can be divided into optical distance sensor, infrared distance sensor, ultrasonic distance sensor and other types according to its working principle. The specific type of the distance sensor set on the robot can be set according to the working scene of the robot, and the embodiment of the present application is not limited to this. The distance sensor is set on the outer surface of the robot, and can measure the interval distance between the surface of the robot and the obstacles in the current environment.
[0078] In step 3013, the tactile sensor of the robot is called to obtain the contact part between the robot and the current environment.
[0079] For example, the tactile sensor is set on the part of the robot that contacts the ground, and the posture and movement mode of the robot can be determined according to the part of the robot that is currently in contact with the ground. For example, the robot is provided with four legs and four wheels, and a tactile sensor is set at the corresponding position of each leg and wheel. If the part of the robot that contacts the ground within the preset time is the four legs, then the robot is in the four-legged movement mode.
[0080] In step 3014, the visual sensor of the robot is called to obtain the position of obstacles in the current environment.
[0081] For example, the visual sensor can be a camera. The camera set on the surface of the robot can determine the specific shape of obstacles near the robot or the specific shape of the terrain. Feature extraction is performed on the image collected by the camera, and combined with the interval distance obtained by the distance sensor, the position of the obstacle in the current environment can be obtained.
[0082] In step 3015, the obstacle position, the contact portion, and the interval distance are used as environmental information of the current environment.
[0083] In an embodiment of the present application, by setting up multiple sensors on the robot and combining the data of the multiple sensors to determine the environmental information and motion parameters around the robot, the parameter acquisition is more comprehensive, which makes it easier to determine the robot's motion mode in the environment based on the environmental information and motion parameters.
[0084] Continue to refer Figure 3A In step 302, the environment type of the current environment is determined based on the environment information.
[0085] For example, the environment type can be set according to the actual application scenario of the robot. The embodiment of the present application is described by taking the application of the robot in the user's life scenario as an example. The environment types of human life scenarios include: upward stairs, downward stairs, uphill, downhill, smooth flat ground, rough flat ground, etc. Assuming that the ground environment is regarded as an obstacle, the terrain can be determined according to the distance between the robot and the ground, and then the current environment is determined according to the terrain. Which environment type is upward stairs, downward stairs, uphill, downhill, smooth flat ground, and rough flat ground.
[0086] In step 303, the current posture of the robot is determined based on the motion parameters of the joints;
[0087] In some embodiments, the motion parameters include angles of joints of the robot; Figure 3A Step 303 in the figure can be implemented in the following manner: based on the angle of the joint and the size of each limb of the robot, determine the relative position between each limb of the robot and the body of the robot; based on the relative position between each limb and the body of the robot, determine the current posture of the robot.
[0088] For example, the relative position between each limb of the robot and the body of the robot can be characterized by the distance between the geometric center of the robot's limb and the geometric center of the body, or by the positional relationship between the limb and the body. For example, the relative position of the limb and the body can be characterized as the geometric center of the limb being at the lower left side of the body and the geometric center of the limb being at the upper right side of the body.
[0089] Each type of posture corresponds to a different preconfigured interval of the relative position between the limbs and the robot's body. The preconfigured interval to which the current relative position between the robot's limbs and the robot's body belongs is obtained, and the posture type corresponding to the preconfigured interval is used as the current posture of the robot.
[0090] In step 304, based on the environmental information and the motion parameters of the joints, the position information of the robot in the current environment is determined.
[0091] For example, the position information of the robot in the current environment includes the center of mass position, joint positions, and contact position of the robot's part in contact with the ground in a coordinate system corresponding to the current environment.
[0092] In some embodiments, the motion parameters include acceleration and angular velocity of the joints of the robot; Figure 3C , Figure 3C It is a flow chart of the robot motion control method provided in the embodiment of the present application. Figure 3A Step 304 in can be Figure 3C Steps 3041 to 3045 in are implemented as described in detail below.
[0093] In step 3041, the angular velocity and acceleration of the joint are integrated respectively to obtain the displacement information of the joint in the current environment.
[0094] For example, the angular velocity is integrated with time to obtain the joint angle change of the click key, including: the absolute value of the joint angle change and the direction of the joint angle change (decrease or increase). The acceleration is integrated with time to obtain the displacement distance, which includes the absolute value of the displacement distance and the direction of the displacement distance change.
[0095] In step 3042, the current position of the robot in the current environment is determined based on the displacement information of the joints, the size of each limb of the robot and the environmental information.
[0096] In some embodiments, the displacement information includes joint angle changes and displacement distances; the environmental information includes terrain data; step 3042 can be implemented in the following manner: performing the following processing on the joint: updating the initial joint angle of the joint according to the joint angle change to obtain the current angle of the joint, updating the initial position of the joint according to the displacement distance to obtain the current position of the joint.
[0097] For example, the angle is determined to increase or decrease according to the direction of the joint angle change. If it decreases, the initial joint angle is subtracted from the absolute value of the joint angle change. If it increases, the initial joint angle is added to the absolute value of the joint angle change to obtain the current angle. Similarly, the initial position is updated according to the displacement distance to obtain the current position.
[0098] In step 3043, the contact position of the part of the robot that contacts the ground in the current environment is determined based on the terrain data.
[0099] For example, the coordinate position of the part of the robot that contacts the ground in the environment is determined based on the terrain data, and the position coordinates of the robot in the coordinate system can be determined by a positioning sensor. Combined with the height of each position on the ground corresponding to the terrain data, the relative position of the part of the robot that contacts the ground and the positioning sensor of the robot, the contact position coordinates of the part of the robot that contacts the ground in the current environment in the coordinate system corresponding to the environment are obtained.
[0100] In step 3044, the center of mass position of the robot in the current environment is determined based on the current angles and current positions of the joints, the size of each of the limbs, and each of the contact positions.
[0101] For example, the current angle and current position of the joint and each of the contact positions are taken as a column in a matrix and combined into a rotation matrix for characterizing the position of the robot in the environment, and the center of mass position of the robot's body in the coordinate system corresponding to the robot is obtained. According to the rotation matrix and the size of each limb of the robot, the center of mass position in the coordinate system corresponding to the robot is updated to the center of mass position in the coordinate system in the current environment.
[0102] In step 3045, the current angle and current position of the joint, each of the contact positions and the center of mass position are taken as the current position of the robot in the current environment.
[0103] In an embodiment of the present application, the position of the robot in the current environment is obtained through multiple aspects, so that the current position of the robot is represented more comprehensively, thereby facilitating motion mode switching based on the current position, thereby avoiding collision of the robot with obstacles during motion mode switching.
[0104] Continue to refer Figure 3A In step 305, in response to the current posture and the position information satisfying the motion mode switching condition, the current motion mode of the robot is switched to the target motion mode corresponding to the environment type.
[0105] For example, the types of motion modes of the robot include at least two of the following: four-wheel mode, four-legged mode, two-wheel mode, two-wheel two-legged mode, two-legged mode, fall self-recovery mode, and folding mode. That is, in the embodiment of the present application, the robot can switch between at least two motion modes, and each motion mode will be explained below.
[0106] In some embodiments, in response to the current posture being a fall posture and the distance between the current position of the robot and the obstacle being greater than a first distance threshold, the current motion mode of the robot is switched to a fall self-recovery mode.
[0107] For example, when at least part of the robot's body is in contact with the ground, the robot is determined to be in a falling posture, and the distance sensor is called to obtain the distance between the outer surface of the robot and the obstacles in the current environment. The first distance threshold may be the minimum distance that the robot will not collide with obstacles during the process of recovering to a standing posture. The fall self-recovery mode refers to a mode in which the robot recovers to a standing or walking state after falling.
[0108] In some embodiments, in response to the current posture being a motion posture and the distance between the current position of the robot and the obstacle being greater than a second distance threshold, the current power of the robot is obtained, and in response to the current power being less than the power threshold, the current motion mode of the robot is switched to a folding mode.
[0109] For example, the folding mode is a movement mode in which the limbs of the robot that are in contact with the ground are folded with the body of the robot. The power threshold can be set to the minimum power to support the movement of the robot. The second distance threshold can be the minimum distance in which the robot will not collide with an obstacle during the process of switching to the folding mode.
[0110] In some embodiments, when the environment type is flat ground, step 305 is implemented in the following manner: in response to the current posture being not a four-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a four-wheel mode adapted to the flat ground.
[0111] For example, the four-wheel mode is a motion mode in which the robot is in contact with the ground with four wheels, and the third distance threshold may be a minimum distance at which the robot will not collide with an obstacle when switching to the four-wheel mode. Figure 4A , Figure 4A is a schematic diagram of the robot structure provided in an embodiment of the present application; Figure 4A The robot moves on four wheels.
[0112] In some embodiments, when the environment type is flat ground, step 305 is implemented in the following manner: in response to the current posture being not a two-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a two-wheel mode adapted to the flat ground.
[0113] refer to Figure 4C , Figure 4C is a schematic diagram of the robot structure provided in an embodiment of the present application; Figure 4C The two-wheel mode is a motion mode in which the part of the robot that contacts the ground is two wheels, and the third distance threshold may be a minimum distance at which the robot will not collide with an obstacle when switching to the two-wheel mode.
[0114] In some embodiments, when the environment type is non-flat ground, step 305 is implemented in the following manner: in response to the current posture being not a quadrupedal movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a quadrupedal mode adapted to the non-flat ground.
[0115] For example, similar to the four-wheel mode, the quadruped mode is a movement mode in which the parts of the robot that are in contact with the ground are four feet, and the third distance threshold can be the minimum distance that the robot will not collide with an obstacle when switching to the quadruped mode.
[0116] For example, the application scenario of the robot is the human living environment. The non-flat environment type can be an environment of going up or down stairs. Figure 4D , Figure 4D is a schematic diagram of the robot structure provided in the embodiment of the present application; Figure 4D The robot climbs the stairs in a four-wheel mode. The inner leg 404 and the outer leg 405 of the robot move alternately, and the motor joints of the inner leg 404 and the outer leg 405 control the limbs to change their angles accordingly, so that the robot moves on the stairs, and the movement of the robot is adapted to the current stair terrain.
[0117] Continue to refer Figure 3A In step 306, the motion parameters of the joints of the robot are configured based on the target motion mode corresponding to the environment type.
[0118] For example, the motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode. The types of motion modes of the robot have been described above. Taking the switching of the bipedal mode to the quadrupedal mode as an example, the acceleration and angular velocity corresponding to the motor joints associated with each foot in contact with the ground in the target motion mode (quadrupedal mode) are obtained, and the acceleration and angular velocity corresponding to the motor joints are converted from digital signals to electrical signals, and the electrical signals are output to the corresponding motor joints so that the motor joints of the robot perform related motions.
[0119] In some embodiments, reference Figure 3D , Figure 3D is a flow chart of a robot motion control method provided in an embodiment of the present application, Figure 3A Step 306 in can be Figure 3D Steps 3061 to 3065 in are implemented as described in detail below.
[0120] In step 3061, the target motion mode corresponding to the environment type is obtained.
[0121] For example, the developers who designed the robot pre-configured the target motion mode corresponding to each environment type. For example, the environment type of going down stairs can be configured as a quadruped mode or a bipedal mode. When the robot is in the state of falling, the fall self-recovery mode is configured.
[0122] In some embodiments, step 3061 can be implemented in the following ways: based on the environmental information corresponding to the environmental type, calling a neural network model to perform feature extraction to obtain environmental features; calling a classifier of the neural network model to classify the environmental features to obtain motion patterns corresponding to the environmental features; and using the motion patterns corresponding to the environmental features as target motion patterns.
[0123] For example, the training sample set of the neural network model includes: different environmental information and the actual environmental type corresponding to the environmental information, and the motion mode corresponding to the actual environmental type. Based on the training sample set, the initialized neural network model is called for training so that the neural network model has the function of classifying environmental information.
[0124] In step 3062, the preconfigured initial motion parameters of the joints of the robot associated with the target motion mode are obtained.
[0125] For example, there are multiple motion modes. Taking the bipedal mode as an example, when the target motion mode of the robot is the bipedal mode, the pre-configured initial motion parameters of the bipedal mode are called, including the angular velocity, acceleration and other parameters of the motor joints associated with the bipeds that are in contact with the ground in the bipedal mode of the robot.
[0126] In step 3063, the initial motion parameters of the joint are iteratively updated according to the environmental information corresponding to the environmental type to obtain multiple target motion parameters.
[0127] For example, the movement of a robot in an environment is affected by the environment, so the initial parameters such as the angular velocity and acceleration of the motor joints are dynamically updated according to the environmental information to obtain the angular velocity and acceleration of the robot at each moment that are adapted to the environmental information, that is, multiple target motion parameters.
[0128] In step 3064, the multiple target motion parameters are combined into a parameter sequence according to the order in which the multiple target motion parameters are generated.
[0129] For example, each target motion parameter in the parameter sequence is used to control the motion state of the joint at different times. The robot includes multiple motor joints, and there are different parameter sequences for the joints.
[0130] In step 3065, the motion state of the joints of the robot is controlled according to the time sequence corresponding to each of the target motion parameters in the parameter sequence.
[0131] For example, the target motion parameters of the parameter sequence are converted from digital signals into electrical signals and output to the joints of the robot to control the joints to execute corresponding motion states, so that the robot can move in the environment.
[0132] The embodiment of the present application determines a target motion mode that adapts to the current environment by acquiring environmental information and the current posture of the robot, and configures the motion parameters of the robot based on the parameters of the target motion mode, so as to switch the robot's motion mode to a motion mode that adapts to the environment, thereby improving the robot's environmental adaptability, enabling the robot to move stably in different types of environments, and enabling the robot to adapt to different application scenarios.
[0133] Below, an exemplary application of the robot motion control method of the embodiment of the present application in a practical application scenario will be described.
[0134] Bipedal mobile robots can adapt to various complex terrains, but they have problems such as low speed and poor stability, which limits the application of bipedal mobile robots in actual human settlement environments. The present application embodiment provides a robot motion control method, which is applied to a robot with multiple motion modes, and can have the advantages of legged robots' multi-terrain adaptability and can also achieve the dynamic performance of the robot's rapid movement.
[0135] For example, the robot in the embodiment of the present application includes a plurality of motion modes, which are distinguished according to the part of the robot contacting the ground, and the motion modes of the robot include: two-wheel balance mode, four-wheel combined with two-foot mode, four-wheel mode, four-wheel combined with four-foot mode, X-wheel mode, two-foot mode, four-foot mode, two-wheel combined with two-foot mode, initial folding mode, and fall self-recovery mode. It can realize movement under different environmental conditions; the mode switching method includes two two-wheel balance modes (including two-wheel balance modes based on inverted pendulum model and two-wheel balance modes based on whole body dynamics control) and two four-wheel modes (including four-wheel joint position control mode and four-wheel joint force control whole body control mode), two two-wheel balance modes and X-wheel mode, initial folding mode and two four-wheel modes, two four-wheel modes and two-wheel combined with two-foot mode, four-wheel mode and two-wheel mode, two-wheel mode and two-foot combined with two-wheel mode, and fall self-recovery mode to other modes.
[0136] Each movement mode of the robot can be regarded as a mode, and the robot in the embodiment of the present application is referred to as a multimodal mobile robot below. The multimodal mobile robot will switch modes according to different terrains, wherein the robot uses the central control system to judge environmental information and switch the corresponding mode after entering a new environment, so that the robot can move in the new environment; the central control system includes an environmental perception module, a central control data processing module, and an execution module. After using the mode switching method, the robot can achieve rapid mode switching in different environments to improve the robot's mobility efficiency and reduce the robot's energy consumption. In addition to a mobile chassis, the robot also includes a waist and upper limbs, the upper limbs are single arms or double arms, and the number of degrees of freedom of the arms is adjustable. The embodiments of the present application are not limited by the specific number of degrees of freedom of the single or double arms and each arm.
[0137] For the convenience of explanation, a type of robot controlled by the robot motion control method of the embodiment of the present application is explained below with reference to the accompanying drawings. Figure 4A , Figure 4A 401, a body part 402, a waist part 403, a leg part 406 (including an inner leg 404 and an outer leg 405), a wheel 407 at the end of each leg, an upper limb 408, a pitch rotation center 409, and a hip rotation center 410. The embodiment of the present application mainly describes the movement of the robot, that is, the movement of the part of the robot that contacts the ground is explained, and the upper limbs are not described in detail here. Figure 4B , Figure 4BIt is a schematic diagram of the robot structure provided in an embodiment of the present application; the robot also includes a side-swing rotation center 501, which is used to rotate the robot body around a direction perpendicular to the ground.
[0138] The motion control device of the robot includes: a sensing and input module (Sensing), a state estimation module (StateEstimation), a path planning module (Path planning), a motion generation and control module (Motion generationand control), a motor low level control module (Motor low level control), and a robot module (Robot).
[0139] Reference 6A, Fig. 6A It is a schematic diagram of the principle of the motion control method of the robot provided in the embodiment of the present application; the sensing and input module (Sensing and Input), the processing time of this module is less than 1 millisecond (except the camera). The hardware output signals contained in this part include: the motor encoder (Encoder) can output the angle q and joint angular velocity qd of all joints, the positioning (GPS) sensor, the tactile sensor of the foot, the force and torque sensor of the foot, the inertia measurement unit (IMU), the camera of the lower body, the distance sensor (TOF), and the remote control and image processor (GUI) responsible for the human-machine interaction input. If the control of the robot's mechanical arm is considered, the torque sensor of the mechanical arm, the tactile sensor at the finger end and the camera of the upper body are also required.
[0140] The signals of each sensor in the sensing and input module are output to the state estimation module. The processing time of the state estimation module is less than 1 millisecond. This module is used to identify the environment and can identify whether the ground is flat and whether there are obstacles around the ground. The position and posture information of the robot base can also be obtained through state estimation. This position and posture can be in the world coordinate system or in the robot's body coordinate system.
[0141] The processing time of the path planning module is less than 1 millisecond, and it is used for global path planning (processing time is less than 1 second) and local path planning (processing time is less than 100 milliseconds). Depending on whether the robot arm is introduced into the control when the robot chassis moves, the global and local path planning are divided into two parts: without the robot arm and with the robot arm.
[0142] The motion generation module has a processing time of less than 10 milliseconds. The motion generation module is divided into two modules: the motion generation module without the robot arm and the motion generation module with the robot arm. Figure 6C , Figure 6C It is a schematic diagram of the modules provided in the embodiment of the present application. The action generation module is used to control the action generation of the switching action (T1~T8) between the two modules, and also includes the action generation of a single mode, such as: two-wheel balance mode, four-wheel + two-foot mode, four-wheel mode, four-wheel + four-foot mode. In the X-wheel mode, its action generation module includes: X-wheel walking, static stair climbing, static stair descent, dynamic stair climbing, and dynamic stair descent. The action generation of the biped (quadruped) mode includes the action generation of the biped (quadruped) walking mode, the biped (quadruped) stair climbing mode, and the biped (quadruped) stair descent mode. The action generation of the two-wheel + biped mode includes the action generation of two-wheel + biped walking, two-wheel + biped climbing stairs, and two-wheel + biped descending stairs. At the same time, the action generation module also includes the action generation of self-recovery from falling in different situations.
[0143] According to the characteristics of the robot, the types of motors in the embodiments of the present application include: wheel motors, leg / waist motors, and mechanical arm motors. These motors control the position or speed or torque input of the joint angles respectively. The control signals required by the motors are all calculated by the motion generation module, and the posture control of the robot can be achieved by sending the motion generation module control signals to the motors.
[0144] Figure 6B It is a module schematic diagram of the state controller provided in an embodiment of the present application; wherein, the two-wheel balancing mode (2-wheel mode) is specifically divided into a two-wheel balancing mode 2-wheel WIP (wheel-inverted pendulum) based on an inverted pendulum model and a two-wheel balancing mode 2-wheel WBC (Whole body control) based on whole body dynamics control, and the two modes can be switched between each other.
[0145] 4-Wheel+Bipedal mode, can walk on flat ground and go up and down stairs.
[0146] Four-wheel driving mode (4-wheel mode), the movement operations performed in this mode include forward, backward, turning, rotating on the spot, etc., which are specifically divided into 4-wheel driving four-wheel joint position control mode (control of the position of other joints of the upper body except the wheels) and four-wheel joint force control whole body control mode 4-wheel WBC, and the two modes can be switched between each other. Figure 4C , Figure 4C is a schematic diagram of the robot structure provided in the embodiment of the present application; Figure 4D , Figure 4D is a schematic diagram of the robot structure provided in the embodiment of the present application; Figure 4C For the dual-wheel mode, Figure 4D The robot in the middle performs stair climbing in four-wheel mode.
[0147] X-wheel mode (X-wheel mode), because in the process of going up the stairs, there will be two-wheel balance mode and four-wheel form mode at the same time, and there will be switching between two-wheel balance mode to four-wheel driving mode, switching between four-wheel driving mode to two-wheel balance mode, switching between two-wheel balance mode and another two-wheel balance mode (left and right wheel switching). At the same time, the ultimate goal of the permutation and combination of these modes is to make the robot's center of mass meet the requirements of the task of going up and down stairs, and the end of the wheel leg is within the specified footprint range. Therefore, this series of possible action permutations and combinations are listed separately and named X-wheel mode (multi-wheel mode).
[0148] Initial Folded mode, power on and off configuration, folded form, complete joint position self-check and other processing in this posture.
[0149] 2-Wheel+Bipedal mode, can walk on flat ground and go up and down stairs.
[0150] Bipedal + Bipedal mode (Bipedal walking mode), can walk on flat ground and go up and down stairs.
[0151] Quadruped walking mode (in the embodiment of the quadruped mode of the present application, each leg has no rotational freedom in the rotational direction), walking is asymmetrical, and the center of mass position can be adjusted forward and backward on the ground with three legs supported, and one leg can be raised to walk on the ground and go up and down stairs. Wheel+Quadruped mode (in the wheel+quadruped mode) can walk on the ground and go up and down stairs.
[0152] Fall recovery mode: The activation of the self-recovery strategy makes the robot try not to hurt humans. While ensuring that humans around it are not injured, it tries to ensure that the environment is not damaged. While ensuring that humans are not injured and the surrounding environment is not damaged, it tries to ensure that its own hardware and software systems are not damaged. The two two-wheel balancing modes and the two four-wheel modes can be switched between each other, and the switching process between modes can be represented by T1&T2; the two two-wheel balancing modes and the X-wheel mode can be switched between each other, and the switching process between modes can be represented by T3&T4; the initial folding mode and the two four-wheel modes can be switched between each other, and the switching process between modes can be represented by T5&T6; the two four-wheel modes and the two-wheel + two-foot mode can be switched between each other, and the switching process between modes can be represented by T7&T8; the four-wheel mode and the two-wheel mode can be switched between each other; the two-wheel mode and the two-foot + two-wheel mode can also be switched between each other. Fall recovery mode can be entered by any other module to start the self-recovery strategy when certain conditions sensed by the robot's own sensors meet certain characteristics, that is, the robot itself is at risk of falling or has already fallen.
[0153] Fig. 6A The meaning of the English signals can be found in Table (1).
[0154]
[0155] The signals and related format information that can be obtained by each specific module in the Sensing and Input section are already in Fig. 6A The detailed description of the information flow is given in the figure. Among them, the motor encoder Encoder gives the angle and angular velocity values of the robot's joint motors. The angle and angular velocity values of the joint motors can determine the overall configuration of the robot, such as the change in the upper body posture angle of the robot in the two-wheel balance state, the length of each leg of the robot in the four-wheel state, the angle between the legs, and the specific angles of the pitch and yaw degrees of freedom of the upper body waist joint; the specific moving direction and moving distance of the robot's center of mass relative to the initial point can be calculated by integration based on the leg and wheel odometers. The positioning sensor can provide relatively accurate global positioning information.
[0156] Inertial measurement units (IMUs) can obtain information such as the robot's displacement and posture in space by integrating acceleration and angular velocity.
[0157] The distance sensor (TOF) can obtain the distance between a certain point on the robot surface and an obstacle for obstacle avoidance. It can also use the relative position and specific changes relative to an unchanging object in an environment for positioning.
[0158] The camera and other visual modules are used to detect and identify objects in the environment, thereby laying the foundation for actions such as mapping and obstacle avoidance. At the same time, they can realize their own positioning based on information such as RGB and Depth.
[0159] For example, the output data of the leg and wheel odometers, GPS, IMU, TOF, camera and other visual modules are combined, and the combined data is input into the neural network model, and the position and posture estimation of the robot can be realized through the neural network model.
[0160] In some embodiments, the output data of the sensor module can be subjected to Kalman filtering, extended Kalman filtering, particle filtering, etc., and the confidence probability of the data of one or more signal sources can be adjusted by adjusting the values of corresponding positions in different parameter matrices under the above algorithm architecture to obtain relatively accurate results.
[0161] By giving data and actual values (ground truth) of multiple data sources under different scenes, different environments and different working conditions of robots, the actual values can come from the collection of motion capture data to train and determine the parameters in the neural network, thereby achieving effective fusion of data from multiple data sources under different scenes, different environments and different working conditions of robots. In some embodiments, an initial value can also be given by a certain signal source based on experience, and then the data of other signal sources can be used to update and iterate on the basis of the initial value. This will form a compromise between computational efficiency and the accuracy of state estimation results to a certain extent.
[0162] The cameras installed on the lower body and upper body of the robot essentially use visual perception to provide the relative position relationship between the robot itself and the external environment. However, the camera on the upper body can also provide relatively accurate local position and posture feedback and environmental information of the robot arm and gripper when performing operation tasks, creating convenient conditions for completing operation tasks based on the upper body design.
[0163] The force and torque sensor (F / T sensor) can obtain the magnitude of the force or torque received by the joint or position where the sensor is installed, and then know the contact and interaction between the robot and the environment. The operation often taken is to introduce the error or relative size of this part of the force into the control closed loop to calculate the control command of the motor at the next moment, so as to achieve the smooth control of the robot joint and the closed-loop smooth interaction with the environment. In a sense, the tactile sensor (Tactile Sensor) has a certain degree of similarity with the function of the force and torque sensor (F / T sensor), but the signal that the tactile sensor can give is more accurate and richer. The tactile sensor generally appears in the form of a sensor array, and the sensor can feedback the contact situation and the magnitude of the contact force at each point. The embodiment of the present application can predict the position of the robot in contact with the external force, the shape of the contact object and the distribution information. The tactile sensor applied to the sole of the foot can more accurately obtain the number of contact points between the robot and the bottom surface, accurately switch the model information used for control of the robot based on the whole body dynamics model (Whole body dynamics control), and the phase information used for planning. The tactile sensors applied to the fingers and palms can more accurately obtain the shape, state and force information of the objects touched by the hands, thereby completing the operation and even the in-hand manipulation task. The specific use principles of the force and torque sensors (F / T sensors) and tactile sensors installed on the upper and lower body are the same, so I will not repeat them here.
[0164] In some embodiments, at the initial stage of movement or when the movement is about to end, the robot needs to be charged or folded. At this time, the perception system needs to fuse various data streams to give whether there are obstacles around the robot, the specific distance of the obstacle to the robot, etc. When it is confirmed that the surrounding environment and the robot's own state meet the state switching conditions, it switches to the initial folding mode; the switching conditions are, for example: determining that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching of the movement mode, and the robot is in the non-folding mode.
[0165] In the scenario where the robot falls down accidentally and needs to recover to the normal motion state sequence while ensuring the safety of itself, the environment and the user, the perception system needs to fuse the data streams to give the existence of obstacles around the robot, the specific distance of the obstacles from the robot, etc. When it is confirmed that the surrounding environment and the robot's own state meet the state switching conditions, it needs to switch to the fall self-recovery mode; the switching conditions include: determining that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching of motion modes.
[0166] In scenarios where the ground is relatively flat, the movement speed is required to be fast, and the robot's running energy consumption is required to be low, the perception system needs to fuse various data streams to give whether there are obstacles around the robot, the specific distance of the obstacle to the robot, etc. When it is confirmed that the surrounding environment and the robot's own state meet the state switching conditions, it will switch to the four-wheel mode first; the switching conditions include: determining that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching of the movement mode, and the robot is in non-four-wheel mode.
[0167] In scenarios where the ground is relatively flat and the robot is required to move at a faster speed, consume less energy, occupy a smaller area, or have higher agility, the perception system needs to fuse the data streams to determine whether there are obstacles around the robot, the specific distance of the obstacle from the robot, etc. When the surrounding environment and the robot's own state meet the state switching conditions, the robot will switch to the dual-wheel mode first; the switching conditions include, for example, ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching of the motion mode, and that the robot is in non-dual-wheel mode.
[0168] In scenarios where the ground is relatively complex, the requirements for movement speed are low, there are no obvious requirements for the robot's operating energy consumption, and the requirements for anti-interference and stability are high, the perception system needs to fuse various data streams to give whether there are obstacles around the robot, the specific distance of the obstacle to the robot, etc. When it is confirmed that the surrounding environment and the robot's own state meet the state switching conditions, it is preferred to switch to the quadruped mode; the switching conditions are, for example: determining that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching process, and the robot is in non-quadruped mode, and the probability of falling during the switching process is less than the preset probability.
[0169] In scenarios where the ground is relatively complex, the requirements for movement speed are low, there are no obvious requirements for the robot's operating energy consumption, and the robot is required to occupy a small area or have high movement sensitivity, the perception system needs to fuse various data streams to give whether there are obstacles around the robot, the specific distance of the obstacle to the robot, etc. When it is confirmed that the surrounding environment and the robot's own state meet the state switching conditions, it is preferred to switch to the quadruped mode; the switching conditions are, for example: determining that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching process, and the robot is in non-quadruped mode, and the probability of falling during the switching process is less than the preset probability.
[0170] When climbing stairs, the perception system needs to integrate various data streams to determine whether there are obstacles around the robot, the specific distance of the obstacles to the robot, etc. When the surrounding environment and the robot's own state meet the state switching conditions, it will switch to X-wheel mode or two-wheel + two-legged mode first; combined with the robot's movement characteristics and application scenarios, it can also choose to switch to four-wheel + two-legged mode, four-wheel + four-legged mode, or two-wheel + two-legged mode.
[0171] In some embodiments, for path planning, limited paths are selected that are shorter, have lower robot energy consumption, and have higher execution stability.
[0172] For example: the robot is currently in a four-wheel state. When there are stairs in front of it, as well as flat land and slopes, the robot can also reach the target point, and the difference between the two is not big. The robot will give priority to flat land and slopes that are more suitable for wheeled passage without switching its own motion mode. The robot is currently in a two-wheel balanced state. When there are stairs in front of it, as well as flat land and slopes, the robot can also reach the target point, and the distance of walking stairs is much shorter than walking on flat land and slopes. The robot will give priority to the easier-to-switch-to stair climbing mode instead of the flat land and slopes with a longer path.
[0173] The embodiment of the present application obtains environmental information and the current posture of the robot to determine the target motion mode that adapts to the current environment, so that the robot's motion mode is switched to a motion mode that adapts to the environment, thereby improving the robot's environmental adaptability, enabling the robot to stably move in different types of environments, and enabling the robot to adapt to different application scenarios. In different motion modes, the motion trajectory generation method is combined with the corresponding control strategy to reduce the difficulty of the robot's movement in a complex environment, thereby improving the robot's environmental adaptability.
[0174] The following is a description of an exemplary structure of a robot motion control device 455 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2As shown, the software modules in the motion control device 455 of the robot stored in the memory 450 may include: a sensing module 4551, configured to obtain environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot; a state estimation module 4552, configured to determine the environmental type of the current environment based on the environmental information; the state estimation module 4552, configured to determine the current posture of the robot based on the motion parameters of the joints; the state estimation module 4552, configured to determine the position information of the robot in the current environment based on the environmental information and the motion parameters of the joints; a motion control module 4553, configured to switch the current motion mode of the robot to a target motion mode corresponding to the environment type in response to the current posture and the position information satisfying a motion mode switching condition; a motor control module 4554, configured to configure the motion parameters of the joints of the robot based on the target motion mode corresponding to the environment type, wherein the motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
[0175] In some embodiments, the motion parameters include acceleration and angular velocity of the joints of the robot; the state estimation module 4552 is configured to integrate the angular velocity and acceleration of the joints respectively to obtain displacement information of the joints in the current environment; based on the displacement information of the joints, the size of each limb of the robot and the environmental information, determine the current position of the robot in the current environment.
[0176] In some embodiments, the displacement information includes joint angle changes and displacement distances; the environmental information includes terrain data; the state estimation module 4552 is configured to perform the following processing for the joint: update the initial joint angle of the joint according to the joint angle change to obtain the current angle of the joint, update the initial position of the joint according to the displacement distance to obtain the current position of the joint; determine the contact position of the part of the robot that contacts the ground in the current environment according to the terrain data; determine the center of mass position of the robot in the current environment based on the current angle and current position of the joint, the size of each limb, and each contact position; use the current angle and current position of the joint, each contact position and the center of mass position as the current position of the robot in the current environment.
[0177] In some embodiments, the motion parameters include the angles of the joints of the robot; the state estimation module 4552 is configured to determine the relative position between each limb of the robot and the body of the robot based on the angles of the joints and the size of each limb of the robot; and determine the current posture of the robot based on the relative position between each limb and the body of the robot.
[0178] In some embodiments, the motor control module 4554 is configured to obtain a target motion mode corresponding to the environment type; obtain pre-configured initial motion parameters of the joints of the robot associated with the target motion mode; iteratively update the initial motion parameters of the joints according to the environmental information corresponding to the environment type to obtain multiple target motion parameters; combine the multiple target motion parameters into a parameter sequence according to the sequence of generation times of the multiple target motion parameters, wherein each of the target motion parameters in the parameter sequence is used to control the motion state of the joint at different times; and control the motion state of the joints of the robot according to the time sequence corresponding to each of the target motion parameters in the parameter sequence.
[0179] In some embodiments, the motor control module 4554 is configured to call a neural network model to perform feature extraction based on environmental information corresponding to the environmental type to obtain environmental features; call a classifier of the neural network model to classify the environmental features to obtain motion patterns corresponding to the environmental features; and use the motion patterns corresponding to the environmental features as target motion patterns.
[0180] In some embodiments, the sensing module 4551 is configured to call the sensors of the joints of the robot to obtain the angular velocity, angle and movement speed of the joints, and use the angular velocity, the angle and the movement speed as motion parameters; call the distance sensor of the robot to obtain the interval distance between the surface of the robot and the obstacles in the current environment; call the tactile sensor of the robot to obtain the contact part of the robot and the current environment; call the visual sensor of the robot to obtain the position of the obstacle in the current environment; use the obstacle position, the contact part and the interval distance as the environmental information of the current environment.
[0181] In some embodiments, the types of motion modes of the robot include at least two of the following: four-wheel mode, four-legged mode, two-wheel mode, two-wheel two-legged mode, two-legged mode, fall self-recovery mode and folding mode.
[0182] In some embodiments, the motion control module 4553 is configured to switch the current motion mode of the robot to a fall self-recovery mode in response to the current posture being a fall posture and the distance between the current position of the robot and the obstacle being greater than a first distance threshold.
[0183] In some embodiments, the motion control module 4553 is configured to obtain the current power of the robot in response to the current posture being a motion posture and the distance between the current position of the robot and the obstacle being greater than a second distance threshold, and switch the current motion mode of the robot to a folding mode in response to the current power being less than the power threshold.
[0184] In some embodiments, the motion control module 4553 is configured to, when the environment type is flat ground, switch the current motion mode of the robot to a four-wheel mode adapted to the flat ground in response to the current posture being not a four-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold.
[0185] In some embodiments, the motion control module 4553 is configured to, when the environment type is flat ground, switch the current motion mode of the robot to a two-wheel mode adapted to the flat ground in response to the current posture being not a two-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold.
[0186] In some embodiments, when the environment type is non-flat ground, the motion control module 4553 is configured to switch the current motion mode of the robot to a quadruped mode adapted to the non-flat ground in response to the current posture being not a quadrupedal movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold.
[0187] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the robot motion control method described in the embodiment of the present application.
[0188] The present application embodiment provides a computer-readable storage medium storing computer executable instructions, wherein the computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will be caused to execute the robot motion control method provided by the present application embodiment, for example, Figure 3A The robot motion control method is shown.
[0189] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.
[0190] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.
[0191] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).
[0192] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.
[0193] To summarize, through the embodiments of the present application, by acquiring environmental information and the current posture of the robot, a target motion mode that adapts to the current environment is determined, and the motion parameters of the robot are configured based on the parameters of the target motion mode, so that the robot's motion mode is switched to a motion mode that adapts to the environment, thereby improving the robot's environmental adaptability, enabling the robot to move stably in different types of environments, and enabling the robot to adapt to different application scenarios.
[0194] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.
Claims
1. A robot motion control method, characterized in that: The method comprises: Acquire environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot; Determine the environment type of the current environment based on the environment information; Determining a current posture of the robot based on motion parameters of the joints; Determine the position information of the robot in the current environment based on the environmental information and the motion parameters of the joints; In response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type; The motion parameters of the joints of the robot are configured based on the target motion mode corresponding to the environment type, wherein the motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
2. The method according to claim 1, characterized in that The motion parameters include acceleration and angular velocity of the joints of the robot; Determining the position information of the robot in the current environment based on the environmental information and the motion parameters of the joints includes: Integrate the angular velocity and acceleration of the joint respectively to obtain the displacement information of the joint in the current environment; Based on the displacement information of the joints, the size of each limb of the robot and the environmental information, the current position of the robot in the current environment is determined.
3. The method according to claim 2, characterized in that The displacement information includes joint angle change and displacement distance; the environmental information includes terrain data; Determining the current position of the robot in the current environment based on the displacement information of the joints, the size of each limb of the robot and the environmental information comprises: The following processing is performed on the joint: updating the initial joint angle of the joint according to the change of the joint angle to obtain the current angle of the joint, and updating the initial position of the joint according to the displacement distance to obtain the current position of the joint; Determine, according to the terrain data, a contact position of the portion of the robot that contacts the ground in the current environment; Determining the center of mass position of the robot in the current environment based on the current angle and current position of the joint, the size of each of the limbs, and each of the contact positions; The current angle and current position of the joint, each of the contact positions and the center of mass position are used as the current position of the robot in the current environment.
4. The method according to claim 1, characterized in that: The motion parameters include angles of joints of the robot; Determining the current posture of the robot based on the motion parameters of the joints includes: Determine the relative position between each limb of the robot and the body of the robot based on the angle of the joint and the size of each limb of the robot; Based on the relative position of each limb to the body of the robot, the current pose of the robot is determined.
5. The method according to claim 1, characterized in that The configuring the motion parameters of the joints of the robot based on the target motion mode corresponding to the environment type includes: Obtaining a target motion mode corresponding to the environment type; Acquire preconfigured initial motion parameters of the joints of the robot associated with the target motion mode; Iteratively updating the initial motion parameters of the joint according to the environmental information corresponding to the environmental type to obtain a plurality of target motion parameters; According to the order of generation time of the multiple target motion parameters, the multiple target motion parameters are combined into a parameter sequence, wherein each of the target motion parameters in the parameter sequence is used to control the motion state of the joint at different times; According to the time sequence corresponding to each target motion parameter in the parameter sequence, the motion state of the joint of the robot is controlled.
6. The method according to claim 5, characterized in that The acquiring the target motion mode corresponding to the environment type includes: Based on the environmental information corresponding to the environmental type, a neural network model is called to perform feature extraction to obtain environmental features; Calling the classifier of the neural network model to classify the environmental features to obtain the motion pattern corresponding to the environmental features; The motion pattern corresponding to the environmental feature is used as the target motion pattern.
7. The method according to claim 1, characterized in that The step of obtaining environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot includes: Calling sensors of joints of the robot to obtain angular velocity, angle and movement speed of the joints, and using the angular velocity, angle and movement speed as movement parameters; Invoking a distance sensor of the robot to obtain a distance between a surface of the robot and an obstacle in the current environment; Invoking a tactile sensor of the robot to obtain a contact portion between the robot and the current environment; Calling the visual sensor of the robot to obtain the position of obstacles in the current environment; The obstacle position, the contact portion and the interval distance are used as environmental information of the current environment.
8. The method according to any one of claims 1 to 7, characterized in that: The types of movement modes of the robot include at least two of the following: a four-wheel mode, a four-legged mode, a two-wheel mode, a two-wheel and two-legged mode, a two-legged mode, a fall self-recovery mode, and a folding mode.
9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: In response to the current posture being a fall posture and the distance between the current position of the robot and the obstacle being greater than a first distance threshold, the current motion mode of the robot is switched to a fall self-recovery mode.
10. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: In response to the current posture being a motion posture and the distance between the current position of the robot and the obstacle being greater than a second distance threshold, the current power of the robot is obtained; in response to the current power being less than the power threshold, the current motion mode of the robot is switched to a folding mode.
11. The method according to any one of claims 1 to 7, characterized in that: When the environment type is flat ground, in response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type includes: In response to the current posture being not a four-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a four-wheel mode adapted to the flat ground.
12. The method according to any one of claims 1 to 7, characterized in that: When the environment type is flat ground, in response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type includes: In response to the current posture being not a two-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a two-wheel mode adapted to the flat ground.
13. The method according to any one of claims 1 to 7, characterized in that: When the environment type is non-flat ground, in response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type includes: In response to the current posture not being a quadruped movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a quadruped mode adapted to the non-flat ground.
14. A robot motion control device, characterized in that: The device comprises: A sensor module configured to obtain environmental information of the current environment in which the robot is located and motion parameters of the joints of the robot; A state estimation module, configured to determine an environment type of the current environment based on the environment information; The state estimation module is configured to determine the current posture of the robot based on the motion parameters of the joints; The state estimation module is configured to determine the position information of the robot in the current environment based on the environmental information and the motion parameters of the joints; a motion control module, configured to switch the current motion mode of the robot to a target motion mode corresponding to the environment type in response to the current posture and the position information satisfying a motion mode switching condition; The motor control module is configured to configure motion parameters of the joints of the robot based on the target motion mode corresponding to the environment type, wherein the motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
15. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; The processor is used to implement the robot motion control method described in any one of claims 1 to 13 when executing the computer executable instructions or computer programs stored in the memory.
16. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the robot motion control method according to any one of claims 1 to 13 is implemented.
17. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the robot motion control method according to any one of claims 1 to 13 is implemented.
Citation Information
Patent Citations
Multifunctional leg-and-wheel combination robot and multi-movement-mode intelligent switching method thereof
CN103786806A
Mode switching method and device for robot
CN107553487A
Wheel-step composite mobile platform control system and control method thereof
CN113419417A
Pipeline detection robot, method and device and storage medium
CN114526399A
Multi-motion-state robot and motion state switching control method thereof
CN115056880A
Cited By
Robot control method and device and electronic equipment
CN116560361A
Control method and system based on biped robot door operation and storage medium
CN119407801A
Control method, system and storage medium based on biped robot door operation
CN119407801B
Robot system and safety control method of robot
CN120422214A
Robot operation method and device, computer equipment and storage medium
CN122018542A
