Environmental sensing devices for mobile robots and mobile robots

By deploying multiple sensors on the front side of the mobile robot to perform data fusion, the problem of insufficient obstacle recognition capability of traditional mobile robots in human-robot mixed environments is solved, achieving more efficient path adjustment and passage capability.

CN123113845APending Publication Date: 2026-11-17BEIJING YOUZHUJU NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510622688.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional mobile robots lack the ability to navigate flexibly in human-machine mixed environments, and have difficulty recognizing both static and dynamic obstacles simultaneously, resulting in low passage efficiency.

Method used

The robot employs multiple sensor fusion technologies, including multiple cameras, LiDAR, and a pair of depth cameras deployed on the front side of the mobile robot. Through data fusion, it enhances its environmental perception capabilities, identifies and adjusts its path to avoid obstacles.

Benefits of technology

It improves the perception range and passage efficiency of mobile robots in complex environments, enabling them to flexibly avoid static and dynamic obstacles and enhancing their applicability in human-robot mixed environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113845A_ABST
    Figure CN123113845A_ABST
Patent Text Reader

Abstract

Embodiments of this disclosure provide an environmental perception device for a mobile robot and a mobile robot. The environmental perception device includes multiple cameras arranged on the front side of the mobile robot in the direction of travel, the cameras being arranged along a width direction perpendicular to the direction of travel, and the fields of view of adjacent cameras partially overlapping; a lidar sensor arranged at the center of the front side of the mobile robot in the width direction, with its field of view at least facing the mobile robot in the direction of travel; and a pair of depth cameras arranged on the front side of the mobile robot, the pair of depth cameras being arranged along the height direction, and the fields of view of the pair of depth cameras at least partially overlapping. This improves the accuracy of the mobile robot's perception of obstacles in the environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The exemplary embodiments disclosed herein relate generally to the field of robotics, and particularly to environmental sensing devices for mobile robots and mobile robots. Background Technology

[0002] Autonomous mobile robots (AMRs) are intelligent robots capable of navigating, avoiding obstacles, and performing tasks autonomously in complex environments. They can perceive environmental information through various sensors and plan paths based on this information to complete various tasks such as material handling, logistics delivery, environmental inspection, and household services. Due to their high flexibility and autonomy, AMRs can be widely used in logistics, warehousing, and service industries. Summary of the Invention

[0003] In a first aspect of this disclosure, an environmental sensing device for a mobile robot is provided. The environmental sensing device includes: a plurality of cameras arranged on the front side of the mobile robot in a direction of travel, the plurality of cameras being arranged along a width direction perpendicular to the direction of travel, and the field of view of adjacent cameras partially overlapping; a lidar sensor arranged at the center of the front side of the mobile robot in the width direction, and its field of view at least facing the mobile robot in the direction of travel; and a pair of depth cameras arranged on the front side of the mobile robot, the pair of depth cameras being arranged along a height direction, and the field of view of the pair of depth cameras at least partially overlapping.

[0004] In some embodiments, the plurality of cameras includes a first camera arranged at the center of the front side in the width direction, and the field of view of the first camera is parallel to the direction of travel.

[0005] In some embodiments, the plurality of cameras include pairs of second cameras, each pair of cameras being symmetrically arranged on both sides of the first camera along the width direction, and the field of view of the second cameras being arranged to deviate from the corresponding side in the width direction by a predetermined angle based on the direction of travel.

[0006] In some embodiments, the fused field of view of the multiple cameras is not less than the first angle.

[0007] In some embodiments, the camera includes an RGB camera.

[0008] In some embodiments, the distance between the lidar and the first camera in the height direction does not exceed a predetermined distance.

[0009] In some embodiments, the field of view of the lidar includes a fan-shaped area with a central angle not less than a second angle.

[0010] In some embodiments, the field of view of the first depth camera, which is positioned above the other in a pair of depth cameras, is tilted downward along the direction of travel, and the field of view of the second depth camera, which is positioned below the other in a pair of depth cameras, is tilted upward along the direction of travel.

[0011] In some embodiments, the environmental sensing device further includes an inertial measurement unit disposed within the mobile robot and adapted to calibrate the center position of the mobile robot.

[0012] According to the environmental perception device for a mobile robot provided in this disclosure, multiple environmental data collected by various sensors arranged on the front side of the mobile robot in its direction of travel are fused together, thereby complementing each other. This improves the mobile robot's perception range of environmental data. The environmental perception device of this disclosure can fuse field-of-view images acquired by multiple cameras with different fields of view, environmental data scanned by LiDAR, and depth images at different heights acquired by a pair of depth cameras, thereby improving the mobile robot's environmental perception capability. This facilitates the mobile robot in adjusting its travel path based on ground obstacles, improving the mobile robot's passage efficiency.

[0013] In a second aspect of this disclosure, a mobile robot is provided. The mobile robot includes a body, including a front side facing the direction of travel; an environmental sensing device according to the first aspect of this disclosure, disposed on the front side of the body and adapted to sense the environment in the direction of travel of the mobile robot; and a travel mechanism coupled to the body, adapted to drive the body to move and avoid obstacles in the environment.

[0014] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0016] Figure 1 A schematic diagram of the front side of a walking robot according to some embodiments of the present disclosure is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] As briefly mentioned earlier, with the continuous development of mobile robots, they can now perform tasks in more complex environments. This requires mobile robots to accurately and promptly identify various obstacles in the environment. In industrial or service scenarios, robots need to deal with both static obstacles (such as fixed objects) and dynamic obstacles (such as moving people). Traditional mobile robots lack the ability to flexibly navigate around obstacles in human-robot mixed environments, and can only achieve simple stopping or basic obstacle avoidance, resulting in poor applicability in dynamic and complex environments.

[0021] The environmental perception device and mobile robot for mobile robots provided according to embodiments of this disclosure solve or at least partially solve the aforementioned problems or other potential problems existing in the conventional technology. The environmental perception device for mobile robots provided according to embodiments of this disclosure fuses multiple environmental data collected by multiple sensors arranged on the front side of the mobile robot in the direction of travel, thereby making the environmental data collected by the multiple sensors complementary. This improves the mobile robot's perception range of environmental data. Specifically, it can fuse field-of-view images acquired by multiple cameras with different fields of view, environmental data scanned by LiDAR, and depth images at different heights acquired by a pair of depth cameras, thereby improving the mobile robot's environmental perception capability. This facilitates the mobile robot in adjusting its travel path according to ground obstacles, improving the mobile robot's passage efficiency.

[0022] Figure 1 A schematic diagram of the front side of a traveling robot according to some embodiments of the present disclosure is shown. Figure 1 As shown, the robot generally includes a chassis 1, a traveling mechanism 2 arranged on the ground-facing side of the chassis 1, and a column 3 coupled to the chassis 1. The column 3 is arranged on the side of the chassis 1 facing away from the ground and is located in front of the chassis 1 along the traveling direction. A main control module is installed inside the chassis 1. The main control module can control the operation of the traveling mechanism 2, so that the traveling mechanism 2 can drive the chassis 1 and the column 3 to move along a predetermined path within the working environment.

[0023] The environmental sensing device is arranged on the front side of the mobile robot along the direction of travel, and generally includes multiple cameras 4, LiDAR 5 and a pair of depth cameras 6.

[0024] Multiple cameras 4 are arranged along a width direction perpendicular to the direction of travel, and the fields of view of two adjacent cameras 4 in the width direction partially overlap. In this way, after acquiring multiple field-of-view images from the multiple cameras 4, the main control module can process these multiple field-of-view images to obtain a fused image based on them. For example, the main control module can crop the field-of-view images based on the overlapping portions and then stitch the cropped images together to obtain the fused image. The resulting fused image has a wider field of view than that of a single camera 4, at least in the width direction.

[0025] In this way, the mobile robot's perception range of the environment can be increased. In some embodiments, the overlap angle of the field of view of two adjacent cameras 4 is not less than a first angle; for example, the field of view of two adjacent cameras 4 overlaps by at least 15°. This ensures that two adjacent field of view images have sufficient overlap, thereby improving the accuracy of field of view fusion. The main control module can more accurately determine dynamic obstacles (e.g., people) in the environment based on the environmental images acquired by the cameras 4, and control the mobile robot to avoid dynamic obstacles.

[0026] In some embodiments, among a plurality of cameras 4 arranged along the width direction, the field of view of the middle first camera 4 is parallel to the direction of travel, so that the first camera 4 can acquire the field of view directly in front of the mobile robot. In some other embodiments, pairs of second cameras 4 arranged at both ends of the first camera 4 along the width direction have their respective field of view deviated from the direction of travel by a predetermined angle along the width direction. For example, the field of view of the second camera 4 has an angle of 30° or 45° with the field of view of the first camera 4. In this way, the second camera 4 can acquire the field of view of the mobile robot on both sides along the direction of travel. Thus, the mobile robot can obtain a larger field of view in the width direction.

[0027] In some embodiments, the environmental sensing device includes three cameras 4. A first camera 4 is arranged in the middle of the front side of the mobile robot along the width direction, and a pair of second cameras 4 are arranged at the two corners of the front side of the mobile robot. In some embodiments, the field of view of each of the pair of second cameras 4 is at an angle of 45° with the field of view of the first camera 4.

[0028] It should be understood that the field of view of camera 4 is a cone-shaped area with camera 4 as the vertex. In some embodiments of this disclosure, the field of view direction of camera 4 refers to the height of the geometric cone corresponding to the cone-shaped area. The angle between the first camera 4 and the second camera 4 can be the angle between the heights of the two geometric cones corresponding to the two cameras 4, or it can be the angle between the projections of the two heights onto the horizontal plane.

[0029] In some other embodiments, the mobile robot may also include more cameras 4, which may be arranged on the circumferential sidewalls of the mobile robot. In this way, the mobile robot obtains a wider field of view; for example, multiple cameras 4 can be evenly distributed around the circumference of the mobile robot, and a 360° annular field of view can be obtained around the mobile robot.

[0030] In some embodiments, camera 4 may include an RGB camera 4. The red, green, and blue color sensors inside the RGB camera 4 can accurately reflect the environment in which the mobile robot is currently located. The main control module can identify and determine obstacles in the environment based on the RGB images acquired by the RGB camera 4.

[0031] The lidar 5 is positioned on the front side of the mobile robot and is adapted to scan a fan-shaped area at the height of the lidar 5. In some embodiments, the central angle of the fan-shaped area scanned by the lidar 5 is not less than a second angle. For example, the scanning range of the lidar 5 may include a fan-shaped area extending 270° forward along the direction of travel.

[0032] Accordingly, the front side of the mobile robot's chassis 1 has a receiving slot for housing the LiDAR 5. The two ends of the receiving slot along its width are flush with the sides of the mobile robot. This reduces the obstruction of the LiDAR 5's scanning range by the mobile robot's casing. The main control module can generate a two-dimensional planar map based on the current environmental data scanned by the LiDAR 5. Based on the two-dimensional planar map, the main control module can further determine the travel path related to the task performed by the mobile robot.

[0033] In some embodiments, the LiDAR 5 is positioned below the camera 4, and the LiDAR 5 is aligned with the first camera 4 in the height direction. In some embodiments, the height distance between the LiDAR 5 and the first camera 4 does not exceed a predetermined distance. This ensures that the environmental data acquired by the LiDAR 5 and the environmental images acquired by the camera 4 are as consistent as possible. The main control module can fuse the environmental data acquired by the LiDAR 5 and the environmental images acquired by the camera 4 to determine the current position of moving obstacles (e.g., people) in the generated two-dimensional planar map. Furthermore, the main control module can update the position of moving obstacles in the two-dimensional planar map in real time based on multiple sets of environmental images acquired by the camera 4, thereby enabling the mobile robot to adjust its movement path to avoid moving obstacles.

[0034] A pair of depth cameras 6 are arranged along the height direction, and their fields of view partially overlap. By fusing the depth images acquired by the pair of depth cameras 6 at different heights, the main control module can further analyze the height of obstacles in the environment and provide depth information of obstacles in three-dimensional space, thus compensating for the shortcomings of the LiDAR 5 and camera 4 in scanning two-dimensional planes.

[0035] In some embodiments, the depth camera 6 can be an RGB-D (RGB-Depth) camera, which can simultaneously capture color images (RGB) and depth information (Depth). Specifically, while acquiring color images, the depth camera 6 can also measure the distance from the object surface to the camera 4 using technologies such as infrared, structured light, time-of-flight (TOF), or binocular vision to generate three-dimensional spatial data.

[0036] In some embodiments, the field of view of the first depth camera 6, positioned above the other in the height direction, is tilted downwards in the direction of travel, while the field of view of the second depth camera 6, positioned below the other in the height direction, is tilted upwards in the direction of travel. This causes the field of view directions of the pair of depth cameras 6 to intersect in front of the mobile robot. After comparing the pair of depth images from the two depth cameras 6, the main control module can determine the specific height of any obstacle in front of the mobile robot.

[0037] Furthermore, the main control module can determine whether there are environmental defects such as ground depressions in front of the mobile robot based on the depth image from the first depth camera 6 located above. The main control module can also determine whether there are suspended obstacles in front of the mobile robot based on the depth image from the second depth camera 6 located below (these suspended obstacles may be higher than the height of camera 4 and / or lidar 5, thus preventing camera 4 and / or lidar 5 from detecting them). Further, the main control module can control the mobile robot to avoid obstacles such as ground depressions and suspended obstacles, improving the mobile robot's mobility.

[0038] In some embodiments, the environmental sensing device further includes an inertial measurement unit (IMU). The IMU is located within the main control module and at the geometric center of the cross-section of the chassis 1 perpendicular to the height direction. In this manner, the IMU can accurately provide the main control module with information about the center position of the mobile robot. By combining the center position of the mobile robot sensed by the IMU with a two-dimensional map, the specific orientation of the mobile robot in the two-dimensional map can be accurately determined. The IMU can periodically refresh the center position of the mobile robot. Furthermore, the two-dimensional map can be continuously updated based on the scanning frequency of the camera 4 and the LiDAR 5. By synchronizing the IMU with at least the camera 4, the LiDAR 5, and the depth camera 6, the main control module can more accurately determine the current position of the mobile robot.

[0039] In some embodiments, the sampling periods or refresh rates of the various sensors in the environmental sensing device differ. For example, the LiDAR 5 has a higher scanning frequency, while the camera 4 has a lower sampling frequency. Therefore, methods such as weighted averaging and Kalman filtering can be used to optimize the data to ensure data consistency.

[0040] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. An environmental sensing device for a mobile robot, comprising: Multiple cameras (4) are arranged on the front side of the mobile robot in the direction of travel. The multiple cameras (4) are arranged along the width direction perpendicular to the direction of travel, and the field of view of two adjacent cameras (4) partially overlap. A lidar (5) is disposed on the front side of the mobile robot at the center of the width direction, and its field of view is at least toward the mobile robot in the direction of travel; and A pair of depth cameras (6) are arranged on the front side of the mobile robot, the pair of depth cameras (6) are arranged along the height direction, and the field of view of the pair of depth cameras (6) at least partially overlap.

2. The environmental sensing device according to claim 1, wherein the plurality of cameras (4) includes a first camera (4) arranged on the front side at the middle in the width direction, and the field of view of the first camera (4) is parallel to the direction of travel.

3. The environmental sensing device according to claim 2, wherein the plurality of cameras (4) includes a pair of second cameras (4), each pair of cameras (4) being symmetrically arranged on both sides of the first camera (4) along the width direction, and the field of view of the second camera (4) being arranged to deviate from the corresponding side of the width direction by a predetermined angle based on the travel direction.

4. The environmental sensing device according to claim 3, wherein the fusion field of view of the plurality of cameras (4) is not less than the first angle.

5. The environmental sensing device according to any one of claims 1-4, wherein the camera (4) comprises an RGB camera (4).

6. The environmental sensing device according to claim 1, wherein the distance between the lidar (5) and the first camera (4) in the height direction does not exceed a predetermined distance.

7. The environmental sensing device according to claim 6, wherein the field of view of the lidar (5) includes a fan-shaped area with a central angle not less than a second angle.

8. The environmental sensing device according to claim 1, wherein the field of view of the first depth camera (6) arranged above the pair of depth cameras (6) is tilted downward along the travel direction, and the field of view of the second depth camera (6) arranged below the pair of depth cameras (6) is tilted upward along the travel direction.

9. The environmental sensing device according to any one of claims 1-4 and 6-8, further comprising: An inertial measurement unit is arranged inside the mobile robot and is adapted to calibrate the center position of the mobile robot.

10. A mobile robot, comprising: The main body includes a front side facing the direction of travel; The environmental sensing device according to any one of claims 1-9 is disposed on the front side of the body and adapted to sense the environment in the direction of travel of the mobile robot; and The traveling mechanism (2) is coupled to the body and is adapted to drive the body to move and avoid obstacles in the environment.