Motion control method for robot, motion control apparatus for robot, electronic device and computer-readable storage medium
By obtaining environmental information and joint motion parameters and switching the motion mode to adapt to the environment, the problems of low travel speed and poor stability of the wheeled foot robot are solved, and stable motion and multi-scene adaptation are achieved in different environments.
Patent Information
- Application Number
- PCT/CN2023/131830
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-29
- Filing Date
- 2023-11-15
- Publication Date
- 2024-12-05
AI Technical Summary
The low travel speed and poor stability of the wheel-footed mobile robot limit its application in human living environments. The existing technology lacks effective robot motion control methods to improve environmental adaptability.
By obtaining information about the environment in which the robot is located and joint motion parameters, determining the current environment type and posture, switching the motion mode to adapt to the environment, and configuring the target motion parameters to optimize the motion mode of the ground contact part.
It improves the robot's environmental adaptability, enables it to move stably in different environments, adapts to a variety of application scenarios, and improves general applicability and motion efficiency.
Smart Images

Figure CN2023131830_05122024_PF_FP_ABST
Abstract
Description
Robot motion control method, robot motion control device, electronic device, and computer-readable storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application is based on the Chinese patent application with application number 202310621872.9 and application date May 29, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field
[0003] The present application relates to computer technology, and in particular to a robot motion control method, a robot motion control device, an electronic device, and a computer-readable storage medium. Background Art
[0004] With the development of robotics technology, various types of robots are increasingly being used in smart factories, smart logistics, smart warehousing and other fields to significantly improve automation and reduce human resource costs. Common mobile robots include legged robots and wheeled robots.
[0005] To enable robots to move flexibly and efficiently, and to adapt to complex terrain environments, wheeled-legged mobile robots are often used. However, the low speed and poor stability of wheeled-legged mobile robots limit their application in human-inhabited environments.
[0006] There is currently no better robot motion control method in related technologies that can improve the robot's environmental adaptability.
[0007] Summary of the Invention
[0008] The embodiments of the present application provide a robot motion control method, a robot motion control device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the robot's environmental adaptability.
[0009] The technical solution of the embodiment of the present application is implemented as follows:
[0010] An embodiment of the present application provides a motion control method for a robot, the method being performed by an electronic device, the method comprising:
[0011] Obtaining environmental information of the current environment in which the robot is located and current motion parameters of the robot's joints;
[0012] determining an environment type of the current environment based on the environment information;
[0013] Determining a current posture of the robot based on current motion parameters of the joints;
[0014] Determining position information of the robot in the current environment based on the environmental information and current motion parameters of the joints;
[0015] In response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type;
[0016] Target motion parameters are configured for the joints of the robot based on the target motion mode corresponding to the environment type, wherein the target motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
[0017] The present invention provides a robot motion control device, comprising:
[0018] a sensing module configured to obtain environmental information of the current environment in which the robot is located and current motion parameters of the joints of the robot;
[0019] a state estimation module, configured to determine an environment type of the current environment based on the environment information;
[0020] The state estimation module is configured to determine the current posture of the robot based on the current motion parameters of the joints;
[0021] The state estimation module is configured to determine the position information of the robot in the current environment based on the environmental information and the current motion parameters of the joints;
[0022] a motion control module configured to switch the current motion mode of the robot to a target motion mode corresponding to the environment type in response to the current posture and the position information satisfying a motion mode switching condition;
[0023] The motor control module is configured to configure target motion parameters for the joints of the robot based on the target motion mode corresponding to the environment type, wherein the target motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
[0024] An embodiment of the present application provides an electronic device, comprising:
[0025] a memory for storing computer-executable instructions;
[0026] The processor is configured to implement the robot motion control method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.
[0027] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the robot motion control method provided in the embodiment of the present application when executed by a processor.
[0028] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, the robot motion control method provided by the embodiment of the present application is implemented.
[0029] The embodiments of the present application have the following beneficial effects:
[0030] By acquiring environmental information and the current motion parameters of the robot's current posture, a target motion mode adapted to the current environment is determined, and based on the parameters of the target motion mode, the robot's current motion parameters are configured as target motion parameters adapted to the target motion mode, so that the robot's motion mode is switched to a motion mode adapted to the environment, thereby improving the robot's environmental adaptability, enabling the robot to move stably in different types of environments, enabling the robot to adapt to different application scenarios, and improving the robot's versatility. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] FIG1 is a schematic diagram of an application mode of a motion control method for a robot provided in an embodiment of the present application;
[0032] FIG2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0033] 3A to 3D are flowcharts of a motion control method for a robot according to an embodiment of the present application;
[0034] 4A to 4D are schematic diagrams of the robot structure provided by an embodiment of the present application;
[0035] 5A and 5B are schematic diagrams of robot motion provided by an embodiment of the present application;
[0036] FIG6A is a schematic diagram showing the principle of a motion control method for a robot provided in an embodiment of the present application;
[0037] FIG6B is a module diagram of a state control machine provided in an embodiment of the present application;
[0038] FIG6C is a schematic diagram of a module provided in an embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0040] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0041] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0042] It should be pointed out that the collection and processing of relevant data in this application (for example, environmental information collected during robot operation and visual content in environmental information) should be strictly in accordance with the requirements of relevant national laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.
[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.
[0044] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.
[0045] 1) Mobile robots, such as legged robots and wheeled robots. To enable flexible and efficient movement and adaptability to complex terrain, wheeled-legged mobile robots are commonly used. Wheeled-legged mobile robots are based on legged robots, but with wheels added to the bottom of their foot mechanisms. This gives them the agility of wheeled robots and the flexibility of legged robots, making them adaptable to a variety of complex terrains.
[0046] 2) Multimodal mobile robots. A multimodal mobile robot can switch between legged and wheeled locomotion. For example, a multimodal mobile robot can switch from two wheels to two legs. The "multimodal" in the name of a multimodal mobile robot refers to the robot's multiple locomotion modes, each named after the limb that supports the motion. Examples of locomotion modes include bipedal locomotion, quadrupedal locomotion, two-wheeled locomotion, and four-wheeled locomotion.
[0047] 3) Convolutional Neural Networks (CNNs) are a type of feed-forward neural network (FNN) with a deep structure that incorporates convolutional computations. They are a representative algorithm for deep learning. CNNs possess representation learning capabilities and can perform shift-invariant classification of input images based on their hierarchical structure.
[0048] 4) Inertial Measurement Unit (IMU): A device that measures an object's three-axis attitude angle (or angular rate) and acceleration. This inertial sensor primarily consists of three accelerometers, an acceleration sensor, three gyroscopes, and a computational circuit. The accelerometers detect the object's acceleration signals along three independent axes in the carrier coordinate system, while the gyroscopes detect the carrier's angular velocity signals relative to the navigation coordinate system. Together, they measure the object's angular velocity and acceleration in three-dimensional space and use this information to determine the object's attitude.
[0049] 5) A distance sensor, also known as a displacement sensor, is a type of sensor used to sense the distance between itself and an object to perform a predetermined function. Distance sensors can be categorized into optical, infrared, and ultrasonic distance sensors, depending on their operating principles.
[0050] 6) Tactile sensors: Tactile sensors are sensors used in robots to simulate the sense of touch. They can be divided into contact sensors, torque sensors, pressure sensors, and slip sensors based on their functions.
[0051] The embodiments of the present application provide a robot motion control method, a robot motion control device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the robot's environmental adaptability.
[0052] The following describes exemplary applications of electronic devices provided by embodiments of the present application. The electronic devices provided by embodiments of the present application can be implemented as laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (for example, mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), vehicle-mounted terminals, virtual reality (VR) devices, augmented reality (AR) devices, and other types of user terminals), and can also be implemented as servers. Below, an exemplary application when the device is implemented as a terminal device will be described.
[0053] Referring to FIG1 , FIG1 is a schematic diagram illustrating an application mode of a robot motion control method provided by an embodiment of the present application. For example, FIG1 involves a server 200, a robot body 100, a network 300, and a terminal device 400. The terminal device 400 is connected to the server 200 via the network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of the two.
[0054] For example, the mechanical structure of the robot body 100 is controlled by the terminal device 400. The terminal device 400 calls the robot motion control method provided in the embodiment of the present application, analyzes the environmental information and motion parameters collected by various sensors set on the robot, determines the corresponding motion mode of the robot in the current environment, and controls the robot to switch to the corresponding motion mode, so that the robot can move in a motion mode that adapts to the current environment.
[0055] The embodiments of the present application can be implemented using database technology. A database, in short, can be considered an electronic filing cabinet that stores electronic files, allowing users to add, query, update, and delete data in these files. A "database" is a collection of data that is stored together in a specific manner, can be shared by multiple users, has minimal redundancy, and is independent of applications.
[0056] A database management system (DBMS) is a computer software system designed for managing databases, typically providing basic functions such as storage, retrieval, security, and backup. DBMSs can be categorized by the database model they support, such as relational or XML (Extensible Markup Language); by the type of computer they support, such as server clusters or mobile phones; by the query language they use, such as SQL or XQuery; by performance priorities, such as maximum scale or maximum speed; or by other classification methods. Regardless of the classification method used, some DBMSs are cross-category, for example, supporting multiple query languages simultaneously.
[0057] The embodiments of the present application can also be implemented through cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, as well as the promotion of search services, social networks, mobile commerce and open collaboration, each item may have its own hash code identification mark in the future, and all of them need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.
[0058] In some embodiments, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The electronic device can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present invention.
[0059] Referring to Figure 2, Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device shown in Figure 2 can be a terminal device. The terminal device 400 includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It will be understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus system 440 in Figure 2.
[0060] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.
[0061] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.
[0062] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.
[0063] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.
[0064] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.
[0065] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;
[0066] A network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include Bluetooth, Wi-Fi, and Universal Serial Bus (USB).
[0067] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);
[0068] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.
[0069] In some embodiments, the apparatus provided in the embodiments of the present application can be implemented in software. FIG2 shows a robot motion control apparatus 455 stored in memory 450. This apparatus can be software in the form of a program or plug-in, and includes the following software modules: a sensing module 4551, a state estimation module 4552, a motion control module 4553, and a motor control module 4554. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.
[0070] The motion control method of the robot provided in the embodiment of the present application will be explained in combination with the exemplary application and implementation of the electronic device provided in the embodiment of the present application.
[0071] The following describes the robot motion control method provided by the embodiments of the present application. As previously mentioned, the electronic device that implements the robot motion control method of the embodiments of the present application can be a terminal device, a server, or a combination of the two. Therefore, the execution entity of each step will not be repeated below.
[0072] Refer to Figure 3A, which is a flow chart of the robot motion control method provided in an embodiment of the present application, which will be explained in conjunction with the steps shown in Figure 3A.
[0073] In step 301, environmental information of the current environment in which the robot is located and current motion parameters of the robot's joints are obtained.
[0074] The robot in the embodiment of the present application has different motion modes and can switch between different motion modes. For the convenience of explanation, the following is an explanation of a type of robot controlled by the motion control method of the robot in the embodiment of the present application in conjunction with the accompanying drawings. Referring to Figure 4A, Figure 4A is a schematic diagram of the robot structure provided by the embodiment of the present application; the robot includes a head 401, a torso 402, a waist 403, legs 406 (including inner legs 404 and outer legs 405), wheels 407 at the end of each leg, upper limbs 408, a pitch rotation center 409 and a hip rotation center 410. In the embodiment of the present application, the movement of the robot is mainly explained, that is, the movement of the part of the robot that contacts the ground is explained, and the upper limbs are not specifically explained here. Referring to Figure 4B, Figure 4B is a schematic diagram of the robot structure provided by the embodiment of the present application; the robot also includes a sway rotation center 501, which is used to rotate the robot's torso around a direction perpendicular to the ground. It should be understood that there are many different types of multimodal mobile robots. The robot motion control method provided in the embodiments of the present application can be applied to different types of robots with motion mode switching capabilities, and is not limited to the robot exemplified in Figure 4A.
[0075] For example, the motion parameters include: angular velocity, angle, acceleration, and movement speed of the robot's motor joints. Step 301 can be implemented by calling multiple sensors installed on the robot. The sensors installed on the robot include: positioning (GPS) sensor, foot tactile sensor, foot force and torque sensor, inertia measurement unit (IMU), distance sensor (TOF), remote control and image processor (GUI) responsible for human-machine interaction input, camera, and visual sensor.
[0076] In some embodiments, referring to FIG3B , FIG3B is a flow chart of the motion control method of the robot provided in an embodiment of the present application. Step 301 in FIG3A can be implemented through steps 3011 to 3015 in FIG3B , which are described in detail below.
[0077] In step 3011, the sensors of the joints of the robot are called to obtain the angular velocity, angle and movement speed of the joints, and the angular velocity, angle and movement speed are used as current movement parameters.
[0078] For example, the sensors of the robot's joints are inertial sensors, which are used to measure the three-axis attitude angle (or angular rate) and acceleration of the object. The inertial sensors installed at the motor joints can obtain the angular velocity, angle and movement speed of the motor joints.
[0079] In step 3012, the distance sensor of the robot is called to obtain the distance between the surface of the robot and the obstacles in the current environment.
[0080] For example, distance sensors can be classified into optical distance sensors, infrared distance sensors, ultrasonic distance sensors, and other types based on their operating principles. The specific type of distance sensor installed on the robot can be set according to the robot's working scenario and is not limited in this embodiment of the present application. The distance sensor is installed on the outer surface of the robot and can measure the distance between the robot surface and obstacles in the current environment.
[0081] In step 3013, the tactile sensor of the robot is called to obtain the contact part between the robot and the current environment.
[0082] For example, tactile sensors are installed on the part of the robot that contacts the ground. The robot's posture and movement can be determined based on the part currently in contact with the ground. The part of the robot that contacts the ground is the part that supports the robot's main body for movement. For example, the four legs of the robot are the part that contacts the ground. When the robot walks, the four legs are not in constant contact with the ground, but are periodically or intermittently lifted from the ground and in contact with the ground.
[0083] For another example, a robot is provided with four legs and four wheels, and a tactile sensor is provided at the corresponding position of each leg and wheel. If the part of the robot that contacts the ground within a preset time is the four legs, the robot is in a four-legged motion mode.
[0084] In step 3014, the robot's visual sensor is called to obtain the location of obstacles in the current environment.
[0085] For example, a visual sensor can be a camera. The camera, mounted on the robot's surface, can determine the specific shape of obstacles or terrain near the robot. Feature extraction is performed on the camera image and combined with the distance measured by the range sensor to determine the location of obstacles in the current environment.
[0086] In step 3015, the obstacle position, contact portion, and separation distance are used as environmental information of the current environment.
[0087] In an embodiment of the present application, by setting up multiple sensors on the robot and combining the data of multiple sensors to determine the environmental information and motion parameters around the robot, the parameter acquisition is made more comprehensive, which facilitates the determination of the robot's motion mode in the environment based on the environmental information and motion parameters.
[0088] In an embodiment of the present application, the robot comprehensively obtains environmental information through multiple sensors. The target motion mode and target motion parameters obtained based on the environmental information and current motion parameters can better adapt to the environment, thereby improving the stability of the robot's movement in the environment, allowing the robot to adapt to a variety of different environments.
[0089] Continuing to refer to FIG. 3A , in step 302 , the environment type of the current environment is determined based on the environment information.
[0090] For example, the environment type can be set based on the actual application scenario of the robot. The embodiments of this application are described using the robot application in the user's daily life scenario as an example. The environment types of human life scenarios include: ascending stairs, descending stairs, uphill, downhill, smooth flat ground, rough flat ground, etc. Assume that the ground environment is regarded as an obstacle. The terrain can be determined based on the distance between the robot and the ground. Then, based on the terrain, it is determined whether the current environment belongs to ascending stairs, descending stairs, uphill, downhill, smooth flat ground, or rough flat ground.
[0091] In step 303, the current posture of the robot is determined based on the current motion parameters of the joints;
[0092] In some embodiments, the motion parameters include the angles of the robot's joints; step 303 in Figure 3A can be implemented by: determining the relative position between each limb of the robot and the robot's body based on the angles of the joints and the size of each limb of the robot; and determining the current posture of the robot based on the relative position between each limb and the robot's body.
[0093] For example, the relative position between each limb of the robot and the torso of the robot can be represented by the distance between the geometric center of the limb and the geometric center of the torso of the robot, or by the positional relationship between the limb and the torso. For example, the relative position of the limb and the torso can be represented by the geometric center of the limb being at the lower left side of the torso and the geometric center of the limb being at the upper right side of the torso.
[0094] Each type of posture corresponds to a different preconfigured interval of the relative position between the limbs and the robot's body. The preconfigured interval to which the current relative position between the robot's limbs and the robot's body belongs is obtained, and the posture type corresponding to the preconfigured interval is used as the current posture of the robot.
[0095] In step 304 , the position information of the robot in the current environment is determined based on the environmental information and the current motion parameters of the joints.
[0096] For example, the position information of the robot in the current environment includes: the center of mass position, joint positions, and contact position of the robot with the ground in the coordinate system corresponding to the current environment.
[0097] In some embodiments, motion parameters include the acceleration and angular velocity of the robot's joints; current motion parameters refer to the motion parameters at the current moment. Referring to FIG3C , FIG3C is a schematic flow chart of the robot motion control method provided in an embodiment of the present application. Step 304 in FIG3A can be implemented by steps 3041 to 3045 in FIG3C , as described in detail below.
[0098] In step 3041, the angular velocity and acceleration of the joint are integrated respectively to obtain the displacement information of the joint in the current environment.
[0099] For example, the angular velocity is integrated over time to obtain the joint angle change of the click key, including: the absolute value of the joint angle change and the direction of the joint angle change (decrease or increase). The acceleration is integrated over time to obtain the displacement distance, including the absolute value of the displacement distance and the direction of the displacement distance change.
[0100] In step 3042 , the current position of the robot in the current environment is determined based on the displacement information of the joints, the size of each limb of the robot, and the environmental information.
[0101] In some embodiments, the displacement information includes joint angle changes and displacement distances; the environmental information includes terrain data; step 3042 can be implemented in the following manner: performing the following processing on the joint: updating the initial joint angle of the joint according to the joint angle change to obtain the current angle of the joint, updating the initial position of the joint according to the displacement distance to obtain the current position of the joint.
[0102] For example, the direction of the joint angle change determines whether the current angle is increasing or decreasing. The direction of the joint angle change can be positive or negative, with a positive direction indicating an increase and a negative direction indicating a decrease. If the angle is decreasing, the absolute value of the joint angle change is subtracted from the initial joint angle. If the angle is increasing, the initial joint angle and the absolute value of the joint angle change are added to obtain the current angle. Similarly, the initial position is updated based on the displacement distance to obtain the current position.
[0103] In step 3043, the contact position of the part of the robot that contacts the ground in the current environment is determined based on the terrain data.
[0104] For example, the coordinate position of the part of the robot that contacts the ground in the environment can be determined based on the terrain data. The position coordinates of the robot in the coordinate system can be determined by the positioning sensor. Combined with the height of each position on the ground corresponding to the terrain data, the relative position of the part of the robot that contacts the ground and the positioning sensor of the robot, the contact position coordinates of the part of the robot that contacts the ground in the current environment in the coordinate system corresponding to the environment are obtained.
[0105] In step 3044, the center of mass position of the robot in the current environment is determined based on the current angles and current positions of the joints, the size of each limb, and each contact position.
[0106] For example, the current angle and current position of the joint and each contact position are taken as a column in the matrix and combined into a rotation matrix for representing the position of the robot in the environment, and the center of mass position of the robot's body in the coordinate system corresponding to the robot is obtained. According to the rotation matrix and the size of each limb of the robot, the center of mass position in the coordinate system corresponding to the robot is updated to the center of mass position in the coordinate system of the current environment.
[0107] In step 3045, the current angle and current position of the joint, each contact position and the center of mass position are taken as the current position of the robot in the current environment.
[0108] In an embodiment of the present application, the position of the robot in the current environment is obtained from multiple aspects, so that the current position of the robot is represented more comprehensively, thereby facilitating motion mode switching based on the current position, and avoiding collisions between the robot and obstacles during motion mode switching.
[0109] Continuing to refer to FIG3A , in step 305 , in response to the current posture and position information satisfying the motion mode switching condition, the current motion mode of the robot is switched to the target motion mode corresponding to the environment type.
[0110] For example, the robot's motion modes include at least two of the following: four-wheel mode, four-legged mode, two-wheel mode, two-wheel and two-legged mode, two-legged mode, fall self-recovery mode, and folding mode. In other words, in the embodiments of the present application, the robot can switch between at least two motion modes, each of which will be explained below.
[0111] In some embodiments, in response to the current posture being a fall posture and the distance between the current position of the robot and the obstacle being greater than a first distance threshold, the current motion mode of the robot is switched to a fall self-recovery mode.
[0112] For example, when at least part of the robot's body is in contact with the ground, the robot is determined to be in a fallen posture, and a distance sensor is used to obtain the distance between the robot's outer surface and obstacles in the current environment. The first distance threshold can be the minimum distance at which the robot can recover from colliding with obstacles while standing. Fall self-recovery mode refers to a mode in which the robot recovers from a fall to a standing or walking state.
[0113] In some embodiments, in response to the current posture being a motion posture and the distance between the robot's current position and the obstacle being greater than a second distance threshold, the robot's current power is obtained, and in response to the current power being less than the power threshold, the robot's current motion mode is switched to a folding mode.
[0114] For example, the folding mode is a movement mode in which the robot's limbs, which are used to contact the ground, are folded with the robot's main body. The power threshold can be set to the minimum power required to support the robot's movement. The second distance threshold can be the minimum distance at which the robot can move without colliding with obstacles while switching to the folding mode.
[0115] In some embodiments, when the environment type is flat ground, step 305 is implemented in the following manner: in response to the current posture not being a four-wheel movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a four-wheel mode adapted to flat ground.
[0116] For example, the four-wheel mode is a motion mode in which the robot's ground-contacting portion is on four wheels. The third distance threshold may be the minimum distance at which the robot can switch to the four-wheel mode without colliding with an obstacle. Referring to Figure 4A , Figure 4A is a schematic diagram of the robot structure provided in an embodiment of the present application; in Figure 4A , the robot moves on four wheels.
[0117] In some embodiments, when the environment type is flat ground, step 305 is implemented in the following manner: in response to the current posture not being a two-wheel movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a two-wheel mode adapted to flat ground.
[0118] Referring to Figure 4C, which is a schematic diagram of the robot structure provided by an embodiment of the present application, Figure 4C illustrates the two-wheel mode. The two-wheel mode is a motion mode in which the robot's ground contact portion is two wheels. The third distance threshold may be the minimum distance at which the robot can switch to the two-wheel mode without colliding with an obstacle.
[0119] In some embodiments, when the environment type is non-flat ground, step 305 is implemented in the following manner: in response to the current posture not being a quadrupedal movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a quadrupedal mode adapted to the non-flat ground.
[0120] For example, similar to the four-wheel mode, the four-legged mode is a movement mode in which the part of the robot that contacts the ground is the four feet. The third distance threshold can be the minimum distance that the robot will not collide with an obstacle when switching to the four-legged mode.
[0121] For example, the robot's application scenario is a human living environment. The non-flat terrain type can be an environment involving climbing or descending stairs. Referring to Figure 4D , Figure 4D is a schematic diagram of the robot structure provided by an embodiment of the present application; in Figure 4D , the robot ascends stairs in a four-wheeled mode. The robot's inner legs 404 and outer legs 405 move alternately, and the motor joints of the inner legs 404 and outer legs 405 correspondingly control the limbs to change angles, allowing the robot to move up the stairs, adapting its motion to the current staircase terrain.
[0122] To facilitate understanding of the robot's movement process, refer to Figures 5A to 5B, which are schematic diagrams of the robot movement provided in the embodiments of the present application. Figures 5A and 5B are side views of the robot in Figure 4D. In stage 1 of Figure 5A, the robot's outer leg 405 remains in place, and the inner leg 404 moves toward the high-level steps of the stairs. In stage 2, the inner leg 404 remains on the high-level steps of the stairs, and the outer leg 405 moves toward the high-level steps of the stairs; in stage 3, the outer leg 405 continues to move until both the inner leg 404 and the outer leg 405 reach the high-level steps. Figure 5B shows the process of the robot going up and down the stairs based on the three-stage movement method of Figure 5.
[0123] Continuing with reference to FIG. 3A , in step 306 , target motion parameters are configured for the joints of the robot based on the target motion mode corresponding to the environment type.
[0124] For example, the target motion parameters are used to switch the portion of the robot in contact with the ground to the ground contact portion in the target motion mode. The types of robot motion modes have been described above. Taking the switch from bipedal mode to quadrupedal mode as an example, the acceleration and angular velocity corresponding to the motor joint associated with each foot in contact with the ground in the target motion mode (quadrupedal mode) are obtained, and the acceleration and angular velocity corresponding to the motor joint are converted from digital signals to electrical signals. The electrical signals are output to the corresponding motor joints to cause the robot's motor joints to perform the relevant motion.
[0125] In some embodiments, referring to FIG3D , FIG3D is a flow chart of the motion control method of the robot provided in an embodiment of the present application. Step 306 in FIG3A can be implemented through steps 3061 to 3065 in FIG3D , which are described in detail below.
[0126] In step 3061, the target motion mode corresponding to the environment type is obtained.
[0127] For example, a robot developer might pre-configure the target motion mode for each environment. For example, the robot might configure the environment for descending stairs to be either quadruped or bipedal. If the robot falls, the robot might also configure a fall recovery mode.
[0128] In some embodiments, step 3061 can be implemented in the following ways: based on the environmental information corresponding to the environment type, calling the neural network model to perform feature extraction to obtain environmental features; calling the classifier of the neural network model to classify the environmental features to obtain the motion pattern corresponding to the environmental features; and using the motion pattern corresponding to the environmental features as the target motion pattern.
[0129] For example, the training sample set for the neural network model includes: different environmental information, the actual environmental types corresponding to the environmental information, and the motion patterns corresponding to the actual environmental types. Based on the training sample set, the initialized neural network model is called for training to enable the neural network model to classify the environmental information.
[0130] The training method can be to obtain the predicted classification result, determine the cross-entropy loss of the neural network model based on the difference between the predicted classification result (motion pattern) and the motion pattern corresponding to the actual environment type in the sample, and perform backpropagation processing on the neural network model based on the cross-entropy loss, gradient update the parameters of the neural network model, and obtain the trained neural network model.
[0131] In step 3062 , the preconfigured initial motion parameters of the robot's joints associated with the target motion mode are obtained.
[0132] For example, there are multiple motion modes. Taking the bipedal mode as an example, when the target motion mode of the robot is the bipedal mode, the pre-configured initial motion parameters of the bipedal mode are called, including the angular velocity, acceleration and other parameters of the motor joints associated with the bipedal parts of the robot that are in contact with the ground in the bipedal mode.
[0133] In step 3063, the initial motion parameters of the joint are iteratively updated according to the environmental information corresponding to the environment type to obtain multiple target motion parameters.
[0134] For example, the robot is affected by the environment when it moves in the environment. Therefore, the initial parameters such as the angular velocity and acceleration of the motor joint are dynamically updated according to the environmental information to obtain the angular velocity and acceleration of the robot at each moment that are adapted to the environmental information, that is, multiple target motion parameters.
[0135] In step 3064, the multiple target motion parameters are combined into a parameter sequence according to the order of their generation times.
[0136] For example, each target motion parameter in the parameter sequence is used to control the motion state of the joint at different moments. The robot includes multiple motor joints, and there are different parameter sequences for each joint.
[0137] In step 3065, the motion state of the robot's joints is controlled according to the time sequence corresponding to each target motion parameter in the parameter sequence.
[0138] For example, the target motion parameters of the parameter sequence are converted from digital signals into electrical signals and output to the joints of the robot to control the joints to execute corresponding motion states, so that the robot can adapt to the environment and move stably.
[0139] The embodiment of the present application determines a target motion mode adapted to the current environment by acquiring environmental information and the current posture of the robot, and configures the robot's motion parameters based on the parameters of the target motion mode, so that the robot's motion mode is switched to a motion mode adapted to the environment, thereby improving the robot's environmental adaptability, enabling the robot to move stably in different types of environments, and enabling the robot to adapt to different application scenarios.
[0140] Below, an exemplary application of the robot motion control method according to an embodiment of the present application in a practical application scenario will be described.
[0141] Bipedal mobile robots can adapt to various complex terrains, but they suffer from low speed and poor stability, which limits their application in real-world human environments. This application provides a robot motion control method that, when applied to a multi-motion-mode robot, combines the advantages of a legged robot's multi-terrain adaptability with the dynamic performance of rapid movement.
[0142] For example, the robot in the embodiment of the present application includes multiple motion modes, which are distinguished according to the part of the robot that contacts the ground. The robot's motion modes include: two-wheel balance mode, four-wheel combined with two-foot mode, four-wheel mode, four-wheel combined with four-foot mode, X-wheel mode, two-foot mode, four-foot mode, two-wheel combined with two-foot mode, initial folding mode, and fall self-recovery mode. It can achieve movement under different environmental conditions; the mode switching method includes switching between two two-wheel balance modes (including a two-wheel balance mode based on an inverted pendulum model and a two-wheel balance mode based on whole-body dynamics control) and two four-wheel modes (including a four-wheel joint position control mode and a four-wheel joint force control whole-body control mode), switching between two two-wheel balance modes and the X-wheel mode, switching between the initial folding mode and two four-wheel modes, switching between two four-wheel modes and the two-wheel combined with two-foot mode, switching between the four-wheel mode and the two-wheel mode, switching between the two-wheel mode and the two-foot combined with two-wheel mode, and switching between the fall self-recovery mode and other modes.
[0143] Each movement mode of the robot can be regarded as a mode, and the robot in the embodiment of the present application is hereinafter referred to as a multimodal mobile robot. The multimodal mobile robot will switch modes according to different terrains, wherein after entering a new environment, the robot uses the central control system to judge the environmental information and switch the corresponding mode so that the robot can move in the new environment; the central control system includes an environmental perception module, a central control data processing module, and an execution module. After using the mode switching method, the robot can achieve rapid mode switching in different environments to improve the robot's movement efficiency and reduce the robot's energy consumption. In addition to a mobile chassis, the robot also includes a waist and upper limbs. The upper limbs are single-arm or double-arm, and the number of degrees of freedom of the arms is adjustable. The embodiment of the present application is not limited by the specific number of degrees of freedom of single or double arms and each arm.
[0144] For ease of explanation, a type of robot controlled by the motion control method of the robot in the embodiment of the present application is explained below in conjunction with the accompanying drawings. Referring to Figure 4A, Figure 4A is a schematic diagram of the robot structure provided in the embodiment of the present application; the robot includes a head 401, a torso 402, a waist 403, legs 406 (including inner legs 404, outer legs 405), wheels 407 at the end of each leg, upper limbs 408, a pitch rotation center 409, and a hip rotation center 410. In the embodiment of the present application, the movement mode of the robot is mainly explained, that is, the movement of the part of the robot that contacts the ground is explained, and the upper limbs are not specifically explained here. Referring to Figure 4B, Figure 4B is a schematic diagram of the robot structure provided in the embodiment of the present application; the robot also includes a lateral rotation center 501, which is used to rotate the body of the robot around a direction perpendicular to the ground.
[0145] The robot's motion control device includes: sensing and input module (Sensing), state estimation module (State Estimation), path planning module (Path Planning), motion generation and control module (Motion Generation and Control), motor low level control module (Motor Low Level Control), and robot module (Robot).
[0146] Referring to 6A, FIG6A is a schematic diagram of the principle of the motion control method of the robot provided in an embodiment of the present application; The serial numbers marked in brackets in FIG6A represent different hardware output signals, and the output order between the hardware output signals is not in particular order. Sensing and input module (Sensing and Input), the processing time of this module is less than 1 millisecond (excluding the camera). The hardware output signals contained in this part include: Motor encoder 601 (Encoder) can output the angle q and joint angular velocity qd of all joints, positioning (GPS) system 602, tactile sensor 603 of the foot, force and torque sensor 604 of the foot, inertia sensor 605 (Inertia Measurement Unit, IMU), base camera 606, distance sensor 607 (TOF), and remote control 608 and image processor 609 (GUI) responsible for human-machine interaction input, etc. If the control of the robot's mechanical arm is considered, the arm torque sensor 610 of the mechanical arm, the finger tactile sensor 611 at the finger end and the upper body camera 612 of the upper body are also required.
[0147] The signals from each sensor in the Sensing and Input module are output to the State Estimation module, which processes data in less than 1 millisecond. This module is used to identify the environment, such as whether the ground is flat and whether there are obstacles around it. State estimation also provides the position and attitude information of the robot base, which can be expressed in either the world coordinate system or the robot's body coordinate system.
[0148] The path planning module, with a processing time of less than 1 millisecond, is used for both global path planning (processing time less than 1 second) and local path planning (processing time less than 100 milliseconds). Depending on whether the robotic arm is involved in control during robot chassis movement, both global and local path planning are divided into two parts: without and with the robotic arm.
[0149] The motion generation module has a processing time of less than 10 milliseconds. The motion generation module is divided into two modules. An action generation module without a robotic arm and an action generation module with a robotic arm. Referring to Figure 6C, Figure 6C is a schematic diagram of the modules provided in an embodiment of the present application. The action generation module 631 is used to control the action generation of the switching action (T1 to T8) between the two modules, and also includes the action generation of a single mode, such as: two-wheel balance mode, four-wheel + two-foot mode, four-wheel mode, four-wheel + four-foot mode. In the X-wheel mode, its action generation module includes: X-wheel walking, static stair climbing, static stair descent, dynamic stair climbing, and dynamic stair descent. The action generation of the biped (quadruped) mode includes the action generation of the biped (quadruped) walking mode, the biped (quadruped) stair climbing mode, and the biped (quadruped) stair descent mode. The action generation of the two-wheel + biped mode includes the action generation of the two-wheel + biped walking, the two-wheel + two-foot stair climbing, and the two-wheel + two-foot stair descent. At the same time, the action generation module 631 is also used to control the action generation of fall self-recovery in different scenarios.
[0150] Based on the robot's characteristics, the motor types used in the embodiments of this application include wheel motors, leg / waist motors, and arm motors. These motors control joint angle position, velocity, or torque input, respectively. The control signals required by the motors are calculated by the motion generation module. Sending these control signals to the motors enables posture control of the robot.
[0151] Figure 6B is a module schematic diagram of the state controller provided in an embodiment of the present application; the state controller is used to control the multi-wheel mode, the initial folding mode and the fall self-recovery mode, among which the two-wheel balancing mode (2-wheel mode) is specifically divided into a two-wheel balancing mode 2-wheel WIP (wheel-inverted pendulum) based on an inverted pendulum model and a two-wheel balancing mode 2-wheel WBC (Whole body control) based on whole-body dynamics control, and the two modes can be switched between each other.
[0152] 4-Wheel+Bipedal mode allows walking on flat ground and going up and down stairs.
[0153] Four-wheel driving mode (4-wheel mode) performs movement operations including forward, backward, turning, and rotating in place. It is specifically divided into 4-wheel driving four-wheel joint position control mode (controlling the position of other upper body joints except the wheels) and 4-wheel joint force control full body control mode (4-wheel WBC), and the two modes can be switched between. Refer to Figure 4C, which is a schematic diagram of the robot structure provided by an embodiment of the present application; refer to Figure 4D, which is a schematic diagram of the robot structure provided by an embodiment of the present application; Figure 4C shows the two-wheel mode, and in Figure 4D, the robot performs stair climbing in four-wheel mode.
[0154] X-wheel mode (X-wheel mode), because during the process of climbing stairs, there will be both two-wheel balancing mode and four-wheel form mode. Previously, there will be switching between two-wheel balancing mode and four-wheel driving mode, switching between four-wheel driving mode and two-wheel balancing mode, and switching between two-wheel balancing mode and another two-wheel balancing mode (left-right wheel switching). At the same time, the ultimate goal of the permutation and combination of these modes is to make the robot's center of mass meet the requirements of the task of going up and down stairs, and the end of the wheel leg is within the specified footprint range. Therefore, this series of possible action permutations and combinations are listed separately and named X-wheel mode (multi-wheel mode).
[0155] Initial Folded mode: power on and off configuration, folded form, in which joint position self-checking and other processes are completed.
[0156] 2-Wheel+Bipedal mode allows walking on flat ground and going up and down stairs.
[0157] Bipedal walking mode: can walk on flat ground and go up and down stairs.
[0158] Quadruped walking mode (in this embodiment, each leg in the quadruped mode lacks rotational freedom) allows for asymmetrical walking. The robot can adjust its center of mass forward and backward while supporting itself on the ground with three legs. With one leg lifted, it can walk on flat ground and ascend and descend stairs. Wheel + Quadruped mode also allows for walking on flat ground and ascending and descending stairs.
[0159] Fall recovery mode activates the self-recovery strategy, ensuring that the robot minimizes harm to humans while protecting the surrounding environment. Furthermore, while protecting both humans and the surrounding environment, the robot also minimizes damage to its own hardware and software systems. The two two-wheel balancing modes and the two four-wheel modes can be switched between, with T1 and T2 representing the switching process. The two two-wheel balancing modes and the X-wheel mode can be switched between, with T3 and T4 representing the switching process. The initial folding mode and the two four-wheel modes can be switched between, with T5 and T6 representing the switching process. The two four-wheel modes and the two-wheel + biped mode can be switched between, with T7 and T8 representing the switching process. The four-wheel mode can be switched between the two-wheel mode and the two-wheel mode, and the two-wheel mode and the biped + two-wheel mode can also be switched between. Fall recovery mode can be entered by any other module to initiate a self-recovery strategy when certain conditions sensed by the robot's own sensors meet certain characteristics, that is, when the robot itself is at risk of falling or has already fallen.
[0160] The meanings of the signals corresponding to the English in FIG6A can be referred to in Table (1).
[0161] The signals and related format information available to each specific module in the sensing and input section are described in detail in the information flow in Figure 6A. The motor encoders provide the angles and angular velocities of the robot's joint motors. These values can determine the robot's overall configuration, such as the change in the robot's upper body posture angle when balanced on two wheels, the length of each leg, the angle between the legs, and the specific angles of the pitch and yaw degrees of freedom of the upper body waist joint when the robot is in a four-wheeled state. The specific movement direction and distance of the robot's center of mass relative to the initial point can be calculated through integration based on the leg and wheel odometers. Positioning sensors can provide relatively accurate global positioning information.
[0162] Inertial measurement units (IMUs) can obtain information such as the robot's displacement and posture in space by integrating acceleration and angular velocity.
[0163] The distance sensor (TOF) can obtain the distance between a point on the robot surface and an obstacle for obstacle avoidance. It can also use the relative position and specific changes of unchanging objects in an environment for positioning.
[0164] The camera and other vision modules are used to detect and identify objects in the environment, laying the foundation for actions such as mapping and obstacle avoidance. At the same time, they can achieve their own positioning based on information such as RGB and Depth.
[0165] For example, the output data of the leg and wheel odometers, GPS, IMU, TOF, camera and other vision modules are combined, and the combined data is input into the neural network model, which can be used to estimate the position and posture of the robot.
[0166] In some embodiments, the output data of the sensing module can be subjected to Kalman filtering, extended Kalman filtering, particle filtering, etc. Under the above algorithm architecture, the confidence probability of the data of one or more signal sources can be adjusted by adjusting the values of the corresponding positions in different parameter matrices, thereby obtaining relatively accurate results.
[0167] By providing data and actual values (ground truth) from multiple data sources in different scenarios, different environments, and different working conditions of robots, the actual values can be collected from motion capture data to train and determine the parameters in the neural network, thereby achieving effective fusion of data from multiple data sources in different scenarios, different environments, and different working conditions of robots. In some embodiments, an initial value can also be given based on a certain signal source that is pre-set based on experience, and then data from other signal sources can be used to update and iterate based on the initial value. This will form a compromise between computational efficiency and the accuracy of state estimation results to a certain extent.
[0168] Cameras mounted on the robot's lower and upper bodies essentially use visual perception to provide information about the robot's relative position to the external environment. However, the upper body camera can also provide relatively accurate local position and posture feedback and environmental information for the robot arm and gripper as they perform tasks, facilitating the completion of tasks based on upper-body designs.
[0169] Force and torque sensors (F / T sensors) can measure the force or torque acting on the joint or location where they are installed, thereby determining the robot's contact and interaction with the environment. A common approach is to incorporate the error or relative magnitude of these forces into the control loop to calculate the next motor control command, thereby achieving smooth control of the robot's joints and closed-loop, smooth interaction with the environment. Tactile sensors, in some sense, function similarly to force and torque sensors (F / T sensors), except that they provide more accurate and richer signals. Tactile sensors typically come in the form of sensor arrays, providing feedback on the contact situation and force magnitude at each point. Embodiments of the present application can predict the location of the robot's contact with external forces, as well as the shape and distribution of the contacted objects. Tactile sensors applied to the soles of the feet can more accurately determine the number of contact points between the robot and the bottom surface, accurately switching the model information used for robot control based on the whole-body dynamics control model and the phase information used for planning. Tactile sensors applied to the fingers and palms can more accurately determine the shape, state, and force of objects touched by the hand, enabling manipulation, even in-hand manipulation. The force and torque sensors (F / T sensors) and tactile sensors installed on the upper and lower body share the same principles and will not be further explained here.
[0170] In some embodiments, at the initial stage of movement or at the end of movement, when the robot needs to be charged or folded, the perception system needs to integrate various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle from the robot. When the surrounding environment and the robot's own state meet the state switching conditions, the robot switches to the initial folded mode. Switching conditions include, for example, ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching of movement modes, and that the robot is in the unfolded mode.
[0171] If the robot accidentally falls and needs to recover to its normal motion sequence while ensuring the safety of itself, the environment, and the user, the perception system must integrate various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle from the robot. Once the robot's surroundings and the robot's own state meet the state transition conditions, it will switch to fall recovery mode. For example, the transition condition may include ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the transition.
[0172] In scenarios where the ground is relatively flat, the robot requires fast movement, and low energy consumption, the perception system needs to integrate various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle to the robot. When both the surrounding environment and the robot's own state meet the state switching conditions, the robot will prioritize switching to four-wheel mode. For example, the switching conditions include ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching motion mode, and that the robot is not in four-wheel mode.
[0173] In scenarios where the ground is relatively flat and the robot requires fast movement speed, low energy consumption, a small footprint, or high agility, the perception system needs to integrate various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle from the robot. When both the surrounding environment and the robot's own state meet the state switching conditions, the robot will prioritize switching to two-wheel mode. For example, the switching conditions include ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching process and that the robot is not in two-wheel mode.
[0174] In scenarios where the terrain is relatively complex, movement speed requirements are low, energy consumption is not significantly limited, and interference resistance and stability are high, the perception system needs to fuse various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle to the robot. When the surrounding environment and the robot's own state meet the state switching conditions, the robot will prioritize switching to quadruped mode. For example, the switching conditions include ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching process, that the robot is in non-quadruped mode, and that the probability of the robot falling during the switching process is less than a preset probability.
[0175] In scenarios where the terrain is relatively complex, movement speed requirements are low, there are no significant requirements for robot operating energy consumption, the robot needs to occupy a small area, or the robot needs to have high motion sensitivity, the perception system needs to integrate various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle to the robot. When it is confirmed that both the surrounding environment and the robot's own state meet the state switching conditions, it will prioritize switching to quadruped mode. Switching conditions include: ensuring that the distance between the obstacle and the robot will not cause the robot to collide with the obstacle during the switching motion mode, and that the robot is in non-quadruped mode, and the probability of falling during the switching process is less than a preset probability.
[0176] When climbing stairs, the perception system needs to integrate various data streams to determine whether there are obstacles around the robot and the specific distance of the obstacle to the robot. Once the surrounding environment and the robot's own state meet the state switching conditions, it will prioritize switching to X-wheel mode or two-wheel + two-legged mode. Depending on the robot's motion characteristics and application scenarios, it can also switch to four-wheel + two-legged mode, four-wheel + four-legged mode, or two-wheel + two-legged mode.
[0177] In some embodiments, for path planning, paths with shorter paths, lower robot energy consumption, and higher execution stability are limitedly selected.
[0178] For example, if the robot is currently in four-wheeled mode and encounters stairs, flat ground, and a slope ahead, both of which can reach the destination, and the distance between the two is not much different, the robot will prioritize the flat ground and slope, which are more suitable for wheeled traversal and do not require switching its motion mode. If the robot is currently in two-wheeled balanced mode and encounters stairs, flat ground, and a slope ahead, both of which can reach the destination, and the distance between the stairs and the slope is much shorter, the robot will prioritize the easier stair climbing mode over the longer flat ground and slope.
[0179] The embodiments of the present application obtain environmental information and the robot's current posture to determine a target motion mode that is suitable for the current environment, thereby switching the robot's motion mode to a motion mode that is suitable for the environment. This improves the robot's environmental adaptability, enables the robot to move stably in different environmental types, and enables the robot to adapt to different application scenarios. Under different motion modes, the motion trajectory generation method, combined with corresponding control strategies, reduces the difficulty of the robot's movement in complex environments, thereby improving the robot's environmental adaptability.
[0180] The following further describes an exemplary structure in which the motion control device 455 of the robot provided in the embodiment of the present application is implemented as a software module. In some embodiments, as shown in Figure 2, the software modules in the motion control device 455 of the robot stored in the memory 450 may include: a sensing module 4551, configured to obtain environmental information of the current environment in which the robot is located and current motion parameters of the joints of the robot; a state estimation module 4552, configured to determine the environmental type of the current environment based on the environmental information; the state estimation module 4552, configured to determine the current posture of the robot based on the current motion parameters of the joints; the state estimation module 4552, configured to determine the position information of the robot in the current environment based on the environmental information and the current motion parameters of the joints; a motion control module 4553, configured to switch the current motion mode of the robot to a target motion mode corresponding to the environment type in response to the current posture and position information satisfying the motion mode switching condition; and a motor control module 4554, configured to configure target motion parameters for the joints of the robot based on the target motion mode corresponding to the environment type, wherein the target motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
[0181] In some embodiments, the current motion parameters include the acceleration and angular velocity of the joints of the robot; the state estimation module 4552 is configured to integrate the angular velocity and acceleration of the joints respectively to obtain the displacement information of the joints in the current environment; based on the displacement information of the joints, the size of each limb of the robot and the environmental information, the current position of the robot in the current environment is determined.
[0182] In some embodiments, the displacement information includes joint angle changes and displacement distances; the environmental information includes terrain data; the state estimation module 4552 is configured to perform the following processing on the joint: update the initial joint angle of the joint according to the joint angle change to obtain the current angle of the joint, update the initial position of the joint according to the displacement distance to obtain the current position of the joint; determine the contact position of the part of the robot that contacts the ground in the current environment according to the terrain data; determine the center of mass position of the robot in the current environment based on the current angle and current position of the joint, the size of each limb, and each contact position; use the current angle and current position of the joint, each contact position and the center of mass position as the current position of the robot in the current environment.
[0183] In some embodiments, the current motion parameters include the angles of the robot's joints; the state estimation module 4552 is configured to determine the relative position between each limb of the robot and the robot's body based on the angles of the joints and the size of each limb of the robot; and determine the current posture of the robot based on the relative position between each limb and the robot's body.
[0184] In some embodiments, the motor control module 4554 is configured to obtain a target motion mode corresponding to the environment type; obtain pre-configured initial motion parameters of the robot's joints associated with the target motion mode; iteratively update the initial motion parameters of the joints according to the environmental information corresponding to the environment type to obtain multiple target motion parameters; combine the multiple target motion parameters into a parameter sequence according to the order of generation time of the multiple target motion parameters, wherein each target motion parameter in the parameter sequence is used to control the motion state of the joint at different times; and control the motion state of the robot's joints according to the time sequence corresponding to each target motion parameter in the parameter sequence.
[0185] In some embodiments, the motor control module 4554 is configured to call the neural network model to perform feature extraction based on the environmental information corresponding to the environment type to obtain environmental features; call the classifier of the neural network model to classify the environmental features to obtain the motion pattern corresponding to the environmental features; and use the motion pattern corresponding to the environmental features as the target motion pattern.
[0186] In some embodiments, the sensing module 4551 is configured to call the sensors of the robot's joints to obtain the angular velocity, angle and movement speed of the joints, and use the angular velocity, angle and movement speed as the current motion parameters; call the robot's distance sensor to obtain the interval distance between the robot's surface and obstacles in the current environment; call the robot's tactile sensor to obtain the contact part of the robot and the current environment; call the robot's visual sensor to obtain the position of obstacles in the current environment; and use the obstacle position, contact part and interval distance as environmental information of the current environment.
[0187] In some embodiments, the types of motion modes of the robot include at least two of the following: four-wheel mode, four-legged mode, two-wheel mode, two-wheel and two-legged mode, two-legged mode, fall self-recovery mode, and folding mode.
[0188] In some embodiments, the motion control module 4553 is configured to switch the robot's current motion mode to a fall self-recovery mode in response to the current posture being a fall posture and the distance between the robot's current position and the obstacle being greater than a first distance threshold.
[0189] In some embodiments, the motion control module 4553 is configured to obtain the current power of the robot in response to the current posture being a motion posture and the distance between the current position of the robot and the obstacle being greater than a second distance threshold, and switch the current motion mode of the robot to a folding mode in response to the current power being less than the power threshold.
[0190] In some embodiments, the motion control module 4553 is configured to switch the robot's current motion mode to a four-wheel mode adapted to flat ground in response to the current posture not being a four-wheel moving posture and the distance between the robot's current position and the obstacle being greater than a third distance threshold when the environment type is flat ground.
[0191] In some embodiments, the motion control module 4553 is configured to switch the robot's current motion mode to a two-wheel mode adapted to flat ground in response to the current posture not being a two-wheel moving posture and the distance between the robot's current position and the obstacle being greater than a third distance threshold when the environment type is flat ground.
[0192] In some embodiments, when the environment type is non-flat ground, the motion control module 4553 is configured to switch the robot's current motion mode to a quadruped mode adapted to the non-flat ground in response to the current posture being not a quadruped movement posture and the distance between the robot's current position and the obstacle being greater than a third distance threshold.
[0193] The present invention provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the robot motion control method described in the present invention.
[0194] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, which stores computer-executable instructions or computer programs. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the robot motion control method provided by an embodiment of the present application, for example, the robot motion control method shown in Figure 3A.
[0195] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.
[0196] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
[0197] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).
[0198] As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.
[0199] To sum up, through the embodiments of the present application, environmental information and the current motion parameters of the robot's current posture are obtained, the target motion mode adapted to the current environment is determined, and based on the parameters of the target motion mode, the robot's current motion parameters are configured as target motion parameters adapted to the target motion mode, so that the robot's motion mode is switched to a motion mode adapted to the environment, thereby improving the robot's environmental adaptability, enabling the robot to move stably in different types of environments, enabling the robot to adapt to different application scenarios, and improving the robot's versatility.
[0200] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.
Claims
1. A robot motion control method, the method being executed by an electronic device, the method include: Acquire environmental information of the current environment in which the robot is located and current motion parameters of the joints of the robot; Determine the environment type of the current environment based on the environment information; Determining a current posture of the robot based on current motion parameters of the joints; Determine the position information of the robot in the current environment based on the environmental information and the current motion parameters of the joints; In response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type; The target motion parameters are configured for the joints of the robot based on the target motion mode corresponding to the environment type, wherein the target motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
2. The method according to claim 1, in, The current motion parameters include acceleration and angular velocity of the joints of the robot; The determining, based on the environmental information and the current motion parameters of the joints, the position information of the robot in the current environment comprises: Integrate the angular velocity and acceleration of the joint respectively to obtain the displacement information of the joint in the current environment; Based on the displacement information of the joints, the size of each limb of the robot and the environmental information, the current position of the robot in the current environment is determined.
3. The method according to claim 2, in, The displacement information includes joint angle change and displacement distance; the environmental information includes terrain data; Determining the current position of the robot in the current environment based on the displacement information of the joints, the size of each limb of the robot and the environmental information comprises: The following processing is performed on the joint: updating the initial joint angle of the joint according to the change of the joint angle to obtain the current angle of the joint, and updating the initial position of the joint according to the displacement distance to obtain the current position of the joint; Determine, according to the terrain data, a contact position of the portion of the robot that contacts the ground in the current environment; Determining the center of mass position of the robot in the current environment based on the current angle and current position of the joint, the size of each of the limbs, and each of the contact positions; The current angle and current position of the joint, each of the contact positions and the center of mass position are used as the current position of the robot in the current environment.
4. The method according to claim 1, in, The current motion parameters include angles of joints of the robot; Determining the current posture of the robot based on the current motion parameters of the joints includes: Determine the relative position between each limb of the robot and the body of the robot based on the angle of the joint and the size of each limb of the robot; Based on the relative position of each limb to the body of the robot, the current pose of the robot is determined.
5. The method according to any one of claims 1 to 4, in, The configuring target motion parameters for the joints of the robot based on the target motion mode corresponding to the environment type includes: Obtaining a target motion mode corresponding to the environment type; Acquire preconfigured initial motion parameters of the joints of the robot associated with the target motion mode; Iteratively updating the initial motion parameters of the joint according to the environmental information corresponding to the environmental type to obtain a plurality of target motion parameters; According to the order of generation time of the multiple target motion parameters, the multiple target motion parameters are combined into a parameter sequence, wherein each of the target motion parameters in the parameter sequence is used to control the motion state of the joint at different times; According to the time sequence corresponding to each target motion parameter in the parameter sequence, the motion state of the joint of the robot is controlled.
6. The method according to claim 5, in, The acquiring the target motion mode corresponding to the environment type includes: Based on the environmental information corresponding to the environmental type, a neural network model is called to perform feature extraction to obtain environmental features; Calling the classifier of the neural network model to classify the environmental features to obtain the motion pattern corresponding to the environmental features; The motion pattern corresponding to the environmental feature is used as the target motion pattern.
7. The method according to any one of claims 1 to 6, in, The step of obtaining the environmental information of the current environment in which the robot is located and the current motion parameters of the joints of the robot includes: Calling sensors of joints of the robot to obtain angular velocity, angle and movement speed of the joints, and using the angular velocity, angle and movement speed as current movement parameters; Invoking a distance sensor of the robot to obtain a distance between a surface of the robot and an obstacle in the current environment; Invoking a tactile sensor of the robot to obtain a contact portion between the robot and the current environment; Calling the visual sensor of the robot to obtain the position of obstacles in the current environment; The obstacle position, the contact portion and the interval distance are used as environmental information of the current environment.
8. The method according to any one of claims 1 to 7, in, The types of movement modes of the robot include at least two of the following: a four-wheel mode, a four-legged mode, a two-wheel mode, a two-wheel and two-legged mode, a two-legged mode, a fall self-recovery mode, and a folding mode.
9. The method according to any one of claims 1 to 8, in, The method further comprises: In response to the current posture being a fall posture and the distance between the current position of the robot and the obstacle being greater than a first distance threshold, the current motion mode of the robot is switched to a fall self-recovery mode.
10. The method according to any one of claims 1 to 9, in, The method further comprises: In response to the current posture being a motion posture and the distance between the current position of the robot and the obstacle being greater than a second distance threshold, the current power of the robot is obtained; in response to the current power being less than the power threshold, the current motion mode of the robot is switched to a folding mode.
11. The method according to any one of claims 1 to 10, in, When the environment type is flat ground, in response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type includes: In response to the current posture being not a four-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a four-wheel mode adapted to the flat ground.
12. The method according to any one of claims 1 to 11, in, When the environment type is flat ground, in response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type includes: In response to the current posture being not a two-wheel moving posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a two-wheel mode adapted to the flat ground.
13. The method according to any one of claims 1 to 12, in, When the environment type is non-flat ground, in response to the current posture and the position information satisfying a motion mode switching condition, switching the current motion mode of the robot to a target motion mode corresponding to the environment type includes: In response to the current posture not being a quadruped movement posture and the distance between the current position of the robot and the obstacle being greater than a third distance threshold, the current motion mode of the robot is switched to a quadruped mode adapted to the non-flat ground.
14. A robot motion control device, the device include: A sensor module configured to obtain environmental information of the current environment in which the robot is located and current motion parameters of the joints of the robot; A state estimation module, configured to determine an environment type of the current environment based on the environment information; The state estimation module is configured to determine the current posture of the robot based on the current motion parameters of the joints; The state estimation module is configured to determine the position information of the robot in the current environment based on the environmental information and the current motion parameters of the joints; a motion control module, configured to switch the current motion mode of the robot to a target motion mode corresponding to the environment type in response to the current posture and the position information satisfying a motion mode switching condition; The motor control module is configured to configure target motion parameters for the joints of the robot based on the target motion mode corresponding to the environment type, wherein the target motion parameters are used to switch the part of the robot in contact with the ground to the ground contact part in the target motion mode.
15. An electronic device, the electronic device include: A memory for storing computer executable instructions; The processor is used to implement the robot motion control method described in any one of claims 1 to 13 when executing the computer executable instructions or computer programs stored in the memory.
16. A computer-readable storage medium storing computer-executable instructions or a computer program, wherein the computer-executable instructions or the computer program, when executed by a processor, implements the robot motion control method according to any one of claims 1 to 13.
17. A computer program product, comprising computer executable instructions or a computer program, wherein the computer executable instructions or the computer program, when executed by a processor, implements the robot motion control method according to any one of claims 1 to 13.
Citation Information
Patent Citations
Wheel-foot switching robot system and control method thereof
CN111497965A
Control method and device of multi-legged robot, electronic equipment and storage medium
CN113515135A
Robot motion control method and device, robot and storage medium
CN114815591A
Robot motion control method and device, robot and storage medium
CN114986526A
Robot based on multi-modal fusion in complex limited environment and operation method
CN115319764A
Cited By
Distributed reinforcement learning control method and system for multi-robot cooperative welding
CN120170759A
Humanoid robot falling recovery control method based on multi-stage course learning
CN120269573A
Robot motion control and state monitoring method and device
CN120395997A
A robot motion control and state monitoring method and device
CN120395997B
Apparatus for controlling robot and method thereof
US20250181073A1
