Multi-mode heavy-load walking-aid robot and control system
By integrating sensors and deep learning algorithms, a multimodal heavy-duty assistive robot has been developed, which solves the problem of insufficient mobility of heavy-duty assistive robots in dynamic environments, realizes intelligent navigation and risk avoidance, and improves handling efficiency and safety.
Patent Information
- Application Number
- CN202511739871.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-01-20
Smart Images
Figure CN121361064A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of robot control, and in particular to a multi-modal heavy-duty walking aid robot and a control system. BACKGROUND
[0002] Heavy-duty robots are widely used in the fields of equipment manufacturing and heavy object carrying due to their higher load capacity. The demand for walking aids for patients with mobility disorders (such as the elderly, the disabled, and postoperative rehabilitation patients) is increasing, and traditional walking aids (such as crutches and pulleys) have single functions. Heavy-duty walking aid robots need to balance high load capacity (supporting user weight and auxiliary equipment) and multi-scene adaptability (indoor and outdoor, flat and rugged terrain) to improve the user's ability to live independently. In heavy industrial scenes such as logistics, construction, mining, and heavy equipment manufacturing, or in disaster relief sites such as earthquakes and fires, manual transportation of heavy objects is inefficient and poses a safety hazard. Heavy-duty robots need to have multi-modal motion capabilities (walking, climbing, and obstacle crossing) and high-reliability control to cope with dynamic and unstructured environments, which can reduce the occurrence of transportation and production accidents.
[0003] Current domestic and foreign technologies mainly focus on heavy-duty industrial robots or light-duty humanoid robots, and there are relatively few studies on heavy-duty walking humanoid robots based on multi-modal fusion perception. SUMMARY
[0004] The present application aims to provide a multi-modal heavy-duty walking aid robot and a control system to solve the technical problems raised in the background.
[0005] In a first aspect, the present application provides a multi-modal heavy-duty walking aid robot, comprising: a robot body including a chassis structure, wheels mounted at the bottom of the chassis structure, a main body connecting mechanism mounted on the chassis structure, and a manipulator mechanism mounted on the main body connecting mechanism, the manipulator mechanism being connected to a drive motor; a sensor system including environment sensors mounted on the main body connecting mechanism and the wheels to collect multi-source environmental data around the robot body, attitude sensors mounted on the wheels to collect attitude data, and torque sensors mounted on the drive motor to collect load data of the manipulator mechanism; a controller in communication with the sensor system and the drive motor, the controller being configured to dynamically adjust a walking path of the robot body and a target position on the walking path according to the multi-source environmental data, and to control the drive motor to cause the manipulator mechanism to perform an action on a target object according to the attitude data and the load data when a path point of the walking path reaches the target position.
[0006] As preferred, the main body connecting mechanism comprises a box-shaped structure or a seat-shaped mechanism, wheels are connected to the bottom of the chassis structure, an upper cover structure is installed on the main body connecting mechanism, and a display screen, a lighting lamp, a control button, a communication interface and a voice interaction module are arranged on the upper cover structure; The main body connecting mechanism is provided with a power module, an MCU chip is used as a control chip to realize power management of the power module, and a SoC system chip is used as a main chip of the controller to realize control of the sensor system, the display screen, the voice interaction module and the driving motor by the interface and the communication protocol.
[0007] As preferred, the controller is connected to the cloud through WIFI to upload the multi-source environmental data collected by the sensor system, the load data collected by the mechanical sensor, the dynamically adjusted walking path and the target position on the walking path to the cloud controller.
[0008] As preferred, the environmental sensor comprises: a laser radar, an image sensor, an infrared sensor and an acoustic sensor.
[0009] As preferred, the attitude sensor comprises an accelerometer and a gyroscope, and the attitude sensor is used to collect attitude data of the wheels to determine a road surface state of the robot body, the road surface state being a slope or a flat ground.
[0010] In the second aspect, the application provides a control system of a multi-modal heavy-load walking aid robot, which is based on the multi-modal heavy-load walking aid robot and comprises: a route planning module, which is used to dynamically adjust a walking path of the robot body and a target position on the walking path according to the multi-source environmental data; a mechanical arm execution control module, which is used to control the mechanical arm mechanism to perform an action on a target object according to the attitude data and the load data when a path point of the walking path reaches the target position. As preferred, the environmental sensor comprises a laser radar, an image sensor, an infrared sensor and an acoustic sensor; dynamically adjusting the walking path of the robot body and the target position on the walking path according to the multi-source environmental data comprises: The laser radar constructs a detection map with the robot body as the center in real time through continuous scanning and a SLAM algorithm, and the position information of the identified object detected and having a distance from the robot body within the radius of the detection map is first registered in a complete map constructed in advance to determine whether it is a target object requiring the robot to perform an action, the position information of the target object being labeled in the complete map in advance, and if it is not a target object, the position information of the potential risk object is marked in the detection map, the radius of the detection map being updated in real time according to the number of potential risk objects, and the radius of the detection map being positively correlated with the number of potential risk objects; The image sensor collects images of the potential risk object according to the position information of the potential risk object, and inputs the collected images into a pre-trained identification model to segment the object range and identify the semantic features of the object type; The infrared sensor and the acoustic sensor perform infrared sensing and acoustic sensing on the potential risk object according to the position information of the potential risk object to obtain infrared sensing features and acoustic sensing features, perform standardization processing on the infrared sensing features and the acoustic sensing features, and then perform weighted summation to obtain a risk feature coefficient, and determine the risk level of the potential risk object according to the risk feature coefficient; The risk distance is determined according to the risk level of the potential risk object, the risk distance being positively correlated with the risk level, and a risk range is drawn on the detection map with the potential risk object as the center and the risk distance as the radius; When the next target position on the walking path appears in the detection map, if the next target position on the walking path is within the risk range of a certain potential risk object, a risk avoidance strategy is determined according to the object type of the potential risk object and the maximum working distance of the robot arm, and the target position on the walking path is adjusted according to the risk avoidance strategy; The walking path is solved in the detection map with the current position of the robot body as the starting point, the adjusted target position as the ending point, the risk range as the constraint condition, and the shortest path length as the optimization target, and if there is no solution under the current constraint condition, the radius of the detection map is increased by a preset proportion until there is a solution.
[0011] Preferably, the object type includes small living things, large living things, small equipment, and large equipment; the division standard of the risk level is determined according to the object type, and the probability of small equipment, large equipment, small living things, and large living things being divided into a high risk level under the same risk feature coefficient increases in turn; The risk avoidance strategy is determined according to the object type of the potential risk object and the maximum working distance of the robot arm, and the target position on the walking path is adjusted according to the risk avoidance strategy to obtain a risk avoidance position, which includes: If the object type of the potential risk object is a small device or a large device, the target position is adjusted with the shortest distance between the target object as the optimization target under the constraint condition of not belonging to the risk range, and it is judged whether the maximum working distance of the mechanical hand mechanism is greater than the distance between the target position and the target object at this time; if greater, a walking path is planned according to the target position, and if less, an alarm signal is sent by the route planning module to remind manual intervention; If the object type of the potential risk object is a small living thing or a large living thing, the target position is first adjusted with the shortest distance between the target object as the optimization target under the constraint condition of not belonging to the risk range, and a walking path is planned, the latest position of the potential risk object is updated every preset time length to adjust the target position again and plan the walking path again, until the robot body finally reaches the target position, if the distance between the target position and the target object at this time does not exceed the maximum working distance of the mechanical hand mechanism, the mechanical hand mechanism directly performs an action on the target object, and if the distance between the target position and the target object at this time exceeds the maximum working distance of the mechanical hand mechanism, an alarm signal is sent by the route planning module to remind manual intervention.
[0012] As a preferred, the control of the driving motor according to the attitude data and the load data to make the mechanical hand mechanism perform an action on the target object when the path point of the walking path reaches the target position comprises: The ground state where the robot body is located is determined according to the attitude data, the rotatable range of the mechanical hand mechanism relative to the main body connecting mechanism is determined under the premise of guaranteeing that the robot body will not fall down and will not interfere with the ground according to the ground state, and the rotating posture of the mechanical hand mechanism to perform an action on the target object is determined in the rotatable range with the minimum load data as the target.
[0013] Compared with the prior art, the present application has the following beneficial effects: The multi-modal heavy-load walking aid robot of the present application can realize multi-modal information fusion, analyze image data by using a deep learning algorithm (target detection, semantic segmentation, etc.), construct a dynamic map in combination with a laser radar, and identify potential risk objects such as living things (for example, small living things such as pets or large living things such as pedestrians), devices (which can be divided into small devices and large devices according to size), and the like that may cause risks. The present application can adopt a corresponding risk avoidance strategy according to the type of the potential risk object to realize intelligent navigation and path planning, realize real-time positioning of the robot body and construction of a detection map (SLAM) based on a laser radar and an image sensor. In combination with a related algorithm for risk avoidance, dynamic path planning and intelligent obstacle avoidance are realized. BRIEF DESCRIPTION OF DRAWINGS
[0014] Figure 1 It is a structural schematic diagram of the multi-modal heavy-load walking aid robot of the embodiment of the present application; Figure 2 is Figure 1 Another perspective structural schematic view; Figure 3 is another mode of the multi-modal heavy-duty walking aid robot of the embodiment of the application, an exploded structural schematic view; Figure 4 is Figure 3 Another structural schematic view; Figure 5 is another mode of the multi-modal heavy-duty walking aid robot of the embodiment of the application, a structural schematic view; Figure 6 is an architecture diagram of the control system of the multi-modal heavy-duty walking aid robot of the embodiment of the application. DETAILED DESCRIPTION
[0015] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only for the purpose of description and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. In the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication between two elements inside. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.
[0016] The present application will be further described below in conjunction with the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot limit the protection scope of the present application.
[0017] In conjunction with Figure 1The embodiment provides a multi-modal heavy-duty walking robot, which comprises a robot body, a sensor system and a controller; the robot body comprises a chassis structure 2, wheels 3 mounted at the bottom of the chassis structure 2, a main body connecting mechanism 1 mounted on the chassis structure 2, and a mechanical hand mechanism 12 mounted on the main body connecting mechanism 1, wherein the mechanical hand mechanism 12 is connected with a driving motor; the sensor system comprises environment sensors mounted on the main body connecting mechanism 1 and the wheels 3 to collect multi-source environment data around the robot body, pressure sensors mounted on the wheels 3 to collect pressure data, a posture sensor to collect posture data, and a torque sensor mounted on the driving motor to collect load data of the mechanical hand mechanism 12. The multi-modal heavy-duty walking robot can be applied to indoor navigation, intelligent obstacle avoidance, rehabilitation training, outdoor travel, intelligent carrying and sorting in a production workshop, carrying heavy objects in a disaster site and a construction site, etc. For example, indoor navigation: automatically avoiding pedestrians in a hospital corridor, accurately reaching a target ward, and intelligently avoiding obstacles to safely perform tasks when carrying heavy objects in a disaster site and a construction site.
[0018] The controller is in communication connection with the sensor system and the driving motor, and is used for dynamically adjusting a walking path of the robot body and a target position on the walking path according to the multi-source environment data, and is used for controlling the driving motor to make the mechanical hand mechanism perform an action on a target object according to the pressure data, the posture data and the load data when a path point of the walking path reaches the target position. The posture sensor comprises an accelerometer and a gyroscope, and is used for collecting posture data of the wheels to determine a road surface state in which the robot body is located, wherein the road surface state is a slope or a flat ground.
[0019] The main body connecting mechanism 1 comprises a box-shaped structure or a seat-shaped mechanism, the chassis structure 2 is connected with the wheels 3 at the bottom, the main body connecting mechanism 1 is provided with an upper cover structural member 4, the upper cover structural member 4 is provided with a display screen 5, an illuminating lamp 7, control buttons 81, a communication interface 91 and a voice interaction module; the main body connecting mechanism 1 and the chassis structure 2 and the upper cover structural member 4 of the mechanical structure part are fixedly connected through screws, the inside of the robot is protected through the main body connecting structure 1, and the robot comprises a motor and a driving module 8 (composed of multiple battery modules and a controller), a communication and safety module 9, a control card 10 (provided with a high-computing-power chip, control and data calculation processing of the whole system), sensors (laser radar image sensors and mounted on the upper cover structural member 4, infrared sensors and acoustic sensors mounted on the wheels 3).
[0020] The main body connecting mechanism 1 is internally provided with a power module, an MCU chip is used as a control chip to realize power management on the power module, the controller uses a SoC system-level chip as a main chip, and the SoC system-level chip and the MCU chip realize control on a sensor system, a display screen, a voice interaction module and a driving motor through an interface and a communication protocol.
[0021] The controller is connected with the cloud through WIFI to upload multi-source environment data collected by the sensor system, load data collected by the mechanical sensor, a dynamically adjusted walking path and a target position on the walking path to the cloud controller.
[0022] The multi-modal heavy-load walking robot in the application has three structural forms, one is a multi-modal heavy-load walking robot with an arm (combined with Figure 1 and Figure 2 ), one is a multi-modal heavy-load walking robot without an arm mainly used for state monitoring, detection and patrol in an industrial or rescue site (combined with Figure 3 and Figure 4 ), and one is a robot mainly used for goods carrying in a warehouse or a production site (combined with Figure 5 ).
[0023] Combined with Figure 6For the structure of the multi-modal heavy-duty walker robot, the frame structure adopts high-strength lightweight materials, the seat design conforms to the stability principle, supports height and angle adjustment, is equipped with a safety belt and a non-slip pad, and adopts a non-slip tire. The driving motor adopts a high-torque brushless DC motor to provide stable power output, the driving mode is double-wheel independent driving, and the braking system adopts electromagnetic and mechanical double braking. The control handle supports manual and automatic switching, force feedback design, and integrates an emergency braking button. The display screen (such as a 7-10 inch LCD or OLED) supports multi-point touch control, and the interface displays real-time information such as maps, battery status, speed, navigation path, etc. AR augmented reality navigation can be selected. The image sensor is configured with a high-definition wide-angle camera, supports night vision function, is used for environment recognition, covers multiple visual angles in front, back and left and right, and improves the environment perception ability. The infrared sensor is used for short-distance obstacle detection and is distributed around the wheels to provide 360° non-corner detection and adapt to low-light environments. The 360° scanning laser radar has a detection distance of 10-30 meters, high precision (millimeter level) and high frequency (10-20 Hz), supports real-time environment modeling, is used for SLAM (Simultaneous Localization and Mapping, simultaneous localization and mapping) and dynamic obstacle avoidance. The voice interaction module supports far-field voice recognition and noise reduction, supports voice interaction, supports voice command recognition (such as "forward" "stop") and voice feedback (through a loudspeaker), integrates natural language processing (NLP) technology to improve interaction fluency. Pressure sensors, torque sensors, and attitude sensors are set as mechanical sensors for monitoring the driving motor load state and inclination angle. The energy storage and power supply battery adopts a high-energy-density lithium-ion battery or a solid-state battery. The communication module integrates Wi-Fi, Bluetooth, and 5G modules to support remote control and data transmission. The application provides USB, HDMI and other expansion interfaces to support external devices, and sets front and rear LED lights for night lighting and signal prompting. It is equipped with anti-toppling wheels or an electronic balance system to prevent wheel rollover.
[0024] The software architecture design of the controller includes ROS 2 (Robot Operating System) or embedded real-time operating system (RTOS) based on low-latency sensor data processing. Modular design: decoupling of each functional module for easy upgrading and expansion (such as adding sensors or algorithms). Fault tolerance mechanism: redundant algorithm design (such as multi-sensor cross-validation) to improve system robustness.
[0025] The multi-modal heavy-duty walking robot of the present application provides data management and remote service, analyzes user behavior, records user daily use data (such as travel path, frequently used functions), and optimizes personalized service (such as automatic recommendation of high-frequency routes) through machine learning. Remote monitoring and maintenance can also be performed, and device status (battery health, sensor abnormalities) can be uploaded to the cloud through Wi-Fi / 5G to realize remote fault diagnosis and firmware upgrade.
[0026] The multi-modal heavy-duty walking robot of the present application can realize multi-modal information fusion: using deep learning algorithms (target detection, semantic segmentation, etc.) to analyze sensor data, combining laser radar point cloud to construct a dynamic map, and identifying living things (such as small living things such as pets or large living things such as pedestrians), devices (which can be divided into small and large devices according to size), and key environmental elements. The present application can realize intelligent navigation and path planning, and realize real-time positioning and high-precision map construction (SLAM) based on laser radar and visual sensors. In combination with related algorithms for risk avoidance, dynamic path planning is realized, intelligent obstacle avoidance is realized, and through the ground pressure data feedback of the mechanical sensor, the wheel motor power is adjusted to adapt to scenarios such as slopes. The present application adopts a driving and motion control system: through PID or model predictive control (MPC) algorithm, the speed and torque of the wheel driving motor are precisely controlled to ensure smooth start and stop and steering of the robot.
[0027] In another specific embodiment, a control system of a multi-modal heavy-duty walking robot is provided, which is based on the multi-modal heavy-duty walking robot to realize, the control system comprising: a route planning module for dynamically adjusting the walking path of the robot body and the target position on the walking path according to the multi-source environmental data; a mechanical arm execution control module for controlling the mechanical arm structure to perform actions on target objects according to the posture data and load data when the path point of the walking path reaches the target position. The environmental sensor includes a laser radar, an image sensor, an infrared sensor, and an acoustic sensor; dynamically adjusting the walking path of the robot body and the target position on the walking path according to the multi-source environmental data includes: Step S1, the laser radar constructs a detection map with the robot body as the center in real time through continuous scanning and a SLAM algorithm, and first registers position information of an identified object whose distance from the robot body is within a radius of the detection map in a complete map constructed in advance to determine whether the object is a target object requiring the robot to perform an action, the position information of the target object being labeled in the complete map in advance, if not, marking the position information of the potential risk object in the detection map, the radius of the detection map being updated in real time according to the number of potential risk objects, the radius of the detection map being positively correlated with the number of potential risk objects; through dynamic adjustment of the radius of the detection map, a dynamic detection range adaptive to the risk situation can be obtained, and potential risk objects can be found as much as possible for subsequent corresponding risk avoidance.
[0028] Step S2, the image sensor collects images of the potential risk object according to the position information of the potential risk object, and inputs the collected images into a pre-trained recognition model to segment the object range and identify semantic features of the object type; specifically, the object type includes small living things, large living things, small equipment and large equipment; whether "small" or "large" is divided according to the object range determined by segmentation according to a preset standard, and the object type is determined by probability regression of a deep learning model.
[0029] Step S3, the infrared sensor and the acoustic sensor perform infrared sensing and acoustic sensing on the potential risk object according to the position information of the potential risk object to obtain infrared sensing features and acoustic sensing features. In a specific embodiment, the infrared sensing features and the acoustic sensing features can also assist in judging the object type. The temperature sensed by the infrared sensor can be compared with the temperature range table of living things and equipment to verify the object type obtained by image recognition. The acoustic sensing features are used to verify the object type obtained by image recognition because the acoustic vibration frequency interval of equipment is quite different from that of living things. Further, the infrared sensing features and the acoustic sensing features are standardized and weighted to obtain a risk feature coefficient. The risk level of the potential risk object is determined according to the risk feature coefficient. Because the higher the infrared sensing heat and the greater the acoustic sensing noise, the higher the instability of the potential risk object, the corresponding risk level should also be higher. The risk level divided by comprehensively considering the infrared sensing features and the acoustic sensing features is more in line with the actual situation. The division standard of the risk level is determined according to the object type (the specific division standard can be obtained by pre-calibration or can be flexibly set according to the specific scene. For example, the division standard in a hospital scene needs to be more stringent than that in a factory scene. The same risk feature coefficient needs to be divided into a higher level in a hospital scene). The probability of small equipment, large equipment, small living things, and large living things being divided into a high risk level under the same risk feature coefficient increases in turn.
[0030] Step S4, the risk distance is determined according to the risk level of the potential risk object. The risk distance is positively correlated with the risk level. The risk distance is the radius of the risk range on the detection map with the potential risk object as the center. The division rule of the risk level and the risk distance is obtained by pre-calibration.
[0031] Step S5, when the next target position on the walking path appears in the detection map, if the next target position on the walking path is in the risk range of a potential risk object, the risk avoidance strategy is determined according to the object type of the potential risk object and the maximum working distance of the mechanical hand structure. The target position on the walking path is adjusted according to the risk avoidance strategy. Specifically, since the potential risk object of the equipment type is usually pre-set and the position is not easy to adjust, and the potential risk object of the living thing type may move by itself and may leave by itself later, the risk avoidance strategies for the equipment type and the living thing type are different. The target position on the walking path is adjusted according to the risk avoidance strategy to obtain a risk avoidance position, which includes: If the object type of the potential risk object is a small device or a large device, the target position is adjusted with the shortest distance between the target object as the optimization target under the constraint condition of not belonging to the risk range, and it is judged whether the maximum working distance of the mechanical hand mechanism is greater than the distance between the target position and the target object at this time; if it is greater, a walking path is planned according to the target position, and if it is smaller, an alarm signal is sent by the route planning module to remind manual intervention. Through this strategy, the target position can be automatically adjusted to realize the normal working of the mechanical hand mechanism, but when the maximum working distance of the mechanical hand mechanism is smaller than the distance between the target position and the target object, it cannot be solved by automatic adjustment, and an alarm is triggered in time to manually intervene.
[0032] If the object type of the potential risk object is a small living thing or a large living thing, the target position is first adjusted with the shortest distance between the target object as the optimization target under the constraint condition of not belonging to the risk range, and a walking path is planned, the latest position of the potential risk object is updated every preset time to adjust the target position again and plan the walking path again, until the robot body finally reaches the target position, if the distance between the target position and the target object at this time does not exceed the maximum working distance of the mechanical hand mechanism, the mechanical hand mechanism directly executes the action on the target object, if the distance between the target position and the target object at this time exceeds the maximum working distance of the mechanical hand mechanism, an alarm signal is sent by the route planning module to remind manual intervention. Through this strategy, the potential risk of the living thing can be avoided by continuous updating, and if it cannot be avoided finally, an alarm is triggered in time to manually intervene.
[0033] Step S6: Taking the current position of the robot body as the starting point and the adjusted target position as the ending point, a walking path is solved in the detection map under the constraint condition of not passing through the risk range and the optimization target of the shortest path length, and if there is no solution under the current constraint condition, the radius of the detection map is increased by a preset proportion until there is a solution. Through the above method, fast and safe operation can be realized under the premise of effectively avoiding risks.
[0034] When the path point of the walking path reaches the target position, the action of the mechanical hand mechanism on the target object is controlled according to the attitude data and the load data, including: determining the ground state where the robot body is located according to the attitude data (the ground pressure can also be judged in combination with the pressure data of the wheels, which will decrease under the influence of the inclination component on the slope), determining the rotatable range of the mechanical hand mechanism relative to the main body connecting mechanism under the premise of ensuring that the robot body will not fall down and will not interfere with the ground (usually the range is smaller in the slope state, and the falling down and ground interference are easy to occur), and determining the rotation attitude of the mechanical hand mechanism for executing the action on the target object in the rotatable range with the minimum load data (as much as possible to reduce the pressure of the driving motor) as the target.
[0035] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, the methods can be tangibly embodied in a machine-readable storage medium having stored thereon instructions that can be used to program a processing system to perform the methods. The machine-readable storage medium can be magnetic (e.g., magnetic disks), optical (e.g., optical discs), electrical (e.g., memory devices), or any combination thereof. Such software can be transmitted using any apparatus adapted to transfer a set of instructions from one place to another. A machine-readable storage medium is a computer-related media that stores computer-readable instructions, data structures, program modules or other data. Examples of computer- readable media include RAM, ROM, EPROM, EEPROM, floppy disks, CD-ROMs, DVD-ROMs, SAS, SCSI, data signals, data streams, and any other medium suitable for storing computer program code. The software can be transmitted using any data transmission techniques, e.g., communication network, wireless network, and the like.
[0036] The present application is described in relation to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It is to be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 means for performing one or more functions specified in the flowchart and / or block diagram block or blocks.
[0037] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 means for performing one or more functions specified in the flowchart and / or block diagram block or blocks.
[0038] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 means for performing one or more functions specified in the flowchart and / or block diagram block or blocks.
[0039] The embodiments of the present application described above are illustrative, and not restrictive, of the present application. Modifications and variations of the embodiments of the present application are possible in light of the above teachings. It is therefore to be understood that changes can be made in the form, details, and / or proportions of any of the disclosed embodiments without departing from the spirit and scope of the application as defined by the appended claims.
Claims
1. A multi-modal heavy-duty walker robot, characterized by, The robot body comprises a chassis structure, wheels mounted at the bottom of the chassis structure, a main body connecting mechanism mounted on the chassis structure, and a mechanical arm mechanism connected with a driving motor mounted on the main body connecting mechanism; The sensor system comprises environment sensors mounted on the main body connecting mechanism and the wheels to collect multi-source environment data around the robot body, attitude sensors mounted on the wheels to collect attitude data, and torque sensors mounted on the driving motor to collect load data of the mechanical arm mechanism; The controller is in communication connection with the sensor system and the driving motor, and is configured to dynamically adjust a walking path of the robot body and a target position on the walking path according to the multi-source environment data, and to control the driving motor to make the mechanical arm mechanism perform an action on a target object according to the attitude data and the load data when a path point of the walking path reaches the target position. The main body connecting mechanism comprises a box-shaped structure or a seat-shaped mechanism, the chassis structure is connected with the wheels at the bottom, and an upper cover structural member is mounted on the main body connecting mechanism, and the upper cover structural member is provided with a display screen, a lighting lamp, control buttons, a communication interface, and a voice interaction module; 2. The multi-modal heavy duty walker robot of claim 1, wherein, The main body connecting mechanism is provided with a power module, an MCU chip is used as a control chip to realize power management of the power module, the controller uses an SoC system-level chip as a main chip, and the SoC system-level chip and the MCU chip control the sensor system, the display screen, the voice interaction module, and the driving motor through an interface and a communication protocol. The controller is connected to the cloud through WIFI to upload the multi-source environment data collected by the sensor system, the load data collected by the mechanical sensor, the dynamically adjusted walking path, and the target position on the walking path to the cloud controller.
3. The multi-modal heavy duty walker robot of claim 1, wherein, The environment sensor comprises:
4. The multi-modal heavy duty walker robot of claim 1, wherein, A laser radar, an image sensor, an infrared sensor, and an acoustic sensor. The attitude sensor comprises an accelerometer and a gyroscope, and is configured to collect attitude data of the wheels to determine a road surface state in which the robot body is located, the road surface state being a slope or a flat ground.
5. The multi-modal heavy duty walker robot of claim 1, wherein, The multi-modal heavy-load walking robot according to any one of claims 1-5 comprises a control system comprising:
6. A control system of a multi-modal heavy duty walker robot, characterized by, A route planning module configured to dynamically adjust a walking path of the robot body and a target position on the walking path according to the multi-source environment data; A mechanical arm execution control module configured to control the mechanical arm mechanism to perform an action on a target object according to the attitude data and the load data when a path point of the walking path reaches the target position. The environment sensor comprises a laser radar, an image sensor, an infrared sensor, and an acoustic sensor; 7. The control system of a multi-modal heavy duty walker robot according to claim 6, wherein, The dynamic adjustment of the walking path of the robot body and the target position on the walking path according to the multi-source environment data comprises: The laser radar constructs a detection map with the robot body as the center in real time through continuous scanning and cooperation with a SLAM algorithm; The position information of the detected object within the detection map radius from the robot body is first registered from the detection map to the pre-constructed complete map to determine whether it is a target object that needs the robot to perform an action; If it is not a target object, the position information of the potential risk object is marked in the detection map, and the radius of the detection map is updated in real time according to the number of potential risk objects, and the radius of the detection map is positively correlated with the number of potential risk objects; The image sensor collects images of the potential risk object according to the position information of the potential risk object, and inputs the collected images into a pre-trained recognition model to segment the object range and recognize the semantic features of the object type; The infrared sensor and the acoustic sensor perform infrared sensing and acoustic sensing processing on the potential risk object according to the position information of the potential risk object to obtain a risk feature coefficient; and determine the risk level of the potential risk object according to the risk feature coefficient; A risk distance is determined according to the risk level of the potential risk object, and the risk distance is positively correlated with the risk level; and a risk range is drawn on the detection map with the potential risk object as the center and the risk distance as the radius; When the next target position on the walking path appears in the detection map, if the next target position on the walking path is within the risk range of a potential risk object, a risk avoidance strategy is determined according to the object type of the potential risk object; and the target position on the walking path is adjusted according to the risk avoidance strategy; The current position of the robot body is taken as the starting point, the adjusted target position is taken as the ending point, the risk range is taken as the constraint condition, and the shortest path length is taken as the optimization target to solve the walking path in the detection map; if there is no solution under the current constraint condition, the radius of the detection map is increased by a preset proportion until there is a solution.
8. The control system of a multi-modal heavy duty walker robot according to claim 7, wherein, The object types include small living things, large living things, small equipment, and large equipment; the risk level is determined according to the object type, and the probability of small equipment, large equipment, small living things, and large living things being classified as a high risk level under the same risk feature coefficient increases in turn; The risk avoidance strategy is determined according to the object type of the potential risk object and the maximum working distance of the robot mechanism, and the target position on the walking path is adjusted to obtain a risk avoidance position according to the risk avoidance strategy, including: If the object type of the potential risk object is small equipment or large equipment, the target position is adjusted with the constraint condition of not belonging to the risk range and the optimization target of the shortest distance between the target object and the potential risk object; it is judged whether the maximum working distance of the robot mechanism is greater than the distance between the adjusted target position and the target object at this time; if it is greater, the walking path is planned according to the target position, and if it is smaller, the route planning module sends an alarm signal to remind manual intervention. If the object type of the potential risk object is a small living object or a large living object, the target position is adjusted with the shortest distance between the target object as the optimization target and the constraint condition of not belonging to the risk range, and a walking path is planned based on the adjusted target position. The latest position of the potential risk object is updated every preset time length to adjust the target position again and plan the walking path again until the robot body reaches the target position at a preset speed. If the distance between the target position and the target object at this time does not exceed the maximum working distance of the mechanical hand structure, the mechanical hand structure directly performs an action on the target object. If the distance between the target position and the target object at this time exceeds the maximum working distance of the mechanical hand structure, the route planning module sends an alarm signal to remind manual intervention.
9. The control system of a multi-modal heavy duty walker robot according to claim 6, wherein, The control of the driving motor according to the attitude data and the load data to make the mechanical hand structure perform an action on the target object when the path point of the walking path reaches the target position includes: determining the ground state where the robot body is located according to the attitude data, determining the rotatable range of the mechanical hand structure relative to the body connecting mechanism under the premise that the robot body will not fall down and will not interfere with the ground according to the ground state, and determining the rotating posture of the mechanical hand structure to perform an action on the target object in the rotatable range with the minimum load data as the target.