Obstacle avoidance method and device of robot, computer equipment and storage medium
By combining visual sensors and deep learning models, the system enables home automation robots to accurately identify obstacles and plan paths in complex environments, solving the problem of poor obstacle avoidance adaptability of existing robots and improving safety and intelligence.
Patent Information
- Application Number
- CN202511524256.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-23
- Publication Date
- 2026-01-16
AI Technical Summary
Existing home automation robots, such as robotic vacuum cleaners and window cleaning robots, cannot effectively utilize image and video data for complex task planning. Furthermore, their sensors are easily affected by environmental factors, resulting in an inability to accurately identify obstacles and hazards. They also exhibit poor adaptability and are unable to safely avoid obstacles in complex environments.
Visual sensors are used to acquire image data, and deep learning models are used for obstacle detection and classification. Obstacle avoidance is achieved through image processing and path planning. Dangerous areas are detected and marked in real time, and the robot replans its path after the user terminal confirms the obstacle.
It improves the robot's obstacle avoidance capabilities in complex environments, enabling it to accurately identify obstacles and hazards, enhances the intelligence of autonomous navigation and path planning, and ensures safe operation.
Smart Images
Figure CN121340256A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of obstacle avoidance technology for robots, specifically to an obstacle avoidance method and apparatus for robots, computer equipment, and storage medium. Background Technology
[0002] Currently, many home automation robots, such as robotic vacuum cleaners and window cleaning robots, mainly rely on basic sensors (such as infrared and ultrasonic sensors) to perform tasks, and cannot effectively utilize image and video data for complex task planning and hazard warnings. Furthermore, obstacles, dangerous items, or other unexpected safety issues may arise during housework, which existing robots typically cannot identify and avoid in a timely manner.
[0003] Currently, home robots have the following drawbacks:
[0004] Poor environmental adaptability: Both infrared and ultrasonic sensors are generally susceptible to environmental factors (such as temperature, humidity, and the material of obstacles). This makes them unstable in dynamic environments (such as home or outdoor environments) and may fail to accurately detect various situations.
[0005] Inability to simultaneously acquire detailed environmental information: Infrared and ultrasonic sensors mostly provide only single pieces of information (such as distance or reflection intensity), and cannot provide details such as the specific shape, size, and material of objects in the environment. This makes it difficult for them to provide sufficient data support when performing complex tasks (such as cleaning, moving objects, etc.).
[0006] Unable to adapt to complex interior layouts: In complex interior layouts (such as multiple objects blocking the view, diverse furniture, etc.), these sensors may not be able to effectively identify and avoid all obstacles, and may easily collide with or miss obstacles. Summary of the Invention
[0007] The purpose of this disclosure is to overcome the shortcomings of the prior art and provide a robot obstacle avoidance method and device, computer equipment and storage medium. The robot obstacle avoidance method can guide the robot to perform household tasks based on environmental information and detect and eliminate possible dangers in real time.
[0008] According to a first aspect of the present disclosure, a method for obstacle avoidance of a robot is provided, comprising the following steps:
[0009] Acquire image data within a target area collected by the acquisition device, wherein the image data carries location information;
[0010] The operating status of the target robot within the target area is determined based on the image data and the location information;
[0011] The presence of obstacles in the target area is determined based on the image data and the operating status of the target robot within the target area.
[0012] If there are obstacles in the target area, the target area will be marked as a prohibited area, and a passable area will be replanned.
[0013] In one embodiment, determining the operating status of the target robot within the target area based on the image data and the location information includes:
[0014] If every frame in the image data is identical, and / or the position information carried by each frame is identical, then the target robot is determined to be malfunctioning.
[0015] In one embodiment, determining whether there are obstacles in the target area based on the image data and the operating status of the target robot within the target area includes:
[0016] Extract key image frames from the image data, and extract features from the key image frames;
[0017] Obstacles are detected and classified using a deep learning model and the features of the key image frames;
[0018] Based on the detection and classification results and the operating status of the target robot within the target area, it is determined whether there are obstacles in the target area.
[0019] In one embodiment, the method further includes:
[0020] If obstacles exist in the target area, the obstacles are marked, and the obstacles are located and filtered out.
[0021] The image containing the obstacle is sent to the user terminal that is bound to the target robot.
[0022] The user terminal is instructed to receive and display an image containing the obstacle;
[0023] The system receives return information sent by the user terminal, the return information being used to indicate that the corresponding obstacle in the image containing the obstacle is identified as an obstacle or a hazard;
[0024] The area where the obstacle is located is marked as a no-run zone based on the information of the obstacle.
[0025] In one embodiment, the obstacle information includes the size of the obstacle, the location of the obstacle, and the distance of the target robot from the obstacle.
[0026] In one embodiment, the method further includes:
[0027] The target robot's mission trajectory is constructed based on the passable area and the prohibited area.
[0028] A task requirement map for the target robot is created based on the target robot's task trajectory and the image data.
[0029] According to a second aspect of the present disclosure, an obstacle avoidance device for a robot is provided, the obstacle avoidance device comprising: an acquisition module, a judgment module, a determination module, and a marking module; wherein,
[0030] The acquisition module is used to acquire image data within the target area collected by the acquisition device, and the image data carries location information;
[0031] The judgment module determines the operating status of the target robot within the target area based on the image data and the location information.
[0032] The determining module determines whether there are obstacles in the target area based on the image data and the operating status of the target robot in the target area;
[0033] The marking module is used to mark the target area as a prohibited area when there are obstacles in the target area, and to re-plan the passable area.
[0034] In one embodiment, the determining module is specifically used for:
[0035] If every frame in the image data is identical, and / or the position information carried by each frame is identical, the target robot is determined to be malfunctioning.
[0036] The determining module is specifically used for:
[0037] Extract key image frames from the image data, and extract features from the key image frames;
[0038] Obstacles are detected and classified using a deep learning model and the features of the key image frames;
[0039] Based on the detection and classification results and the operating status of the target robot within the target area, it is determined whether there are obstacles in the target area.
[0040] According to a third aspect of the present disclosure, a computer device is provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.
[0041] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the method as described in any of the above.
[0042] This disclosure provides an obstacle avoidance method for robots, which can guide robots to perform household tasks based on environmental information and detect and eliminate potential dangers in real time. Attached Figure Description
[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0044] Figure 1 This is a schematic diagram illustrating the workflow of an obstacle avoidance system for a robot, provided as an embodiment of this disclosure.
[0045] Figure 2 This is a flowchart illustrating an obstacle avoidance method for a robot provided in an embodiment of this disclosure.
[0046] Figure 3 This is a flowchart illustrating the data acquisition process in this embodiment.
[0047] Figure 4 This is a flowchart illustrating an obstacle avoidance method for a robot provided in an embodiment of this disclosure.
[0048] Figure 5 This is a flowchart illustrating an obstacle avoidance method for a robot provided in an embodiment of this disclosure.
[0049] Figure 6 This is a flowchart illustrating a robot obstacle avoidance method provided in an embodiment of the present disclosure.
[0050] Figure 7 This is an architectural diagram of an obstacle avoidance device for a robot provided in an embodiment of this disclosure.
[0051] Figure 8 This is a schematic diagram of the robot in this embodiment.
[0052] Figure 9 This is an architectural diagram of a computer device provided in an embodiment of the present disclosure. Detailed Implementation
[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0054] Robotic vacuum cleaners often get stuck in a corner or near a power source. In this embodiment, visual sensors (camera, laser, IMU, and GPS) identify and record obstacles in the area, and transmit the surrounding images to the user terminal in real time, providing the user with a live view. The user can switch to manual mode to control the robotic vacuum cleaner to quickly leave the area, and use image recognition to mark the area as a no-operation zone.
[0055] Specifically, the visual sensor in this embodiment has the following characteristics:
[0056] 1. High-precision and detailed environmental perception:
[0057] Strong image perception capabilities: Visual sensors can provide rich image information, capturing detailed features such as the shape, color, texture, and size of objects. Compared with infrared and ultrasonic sensors, visual sensors can more accurately identify complex objects and environments.
[0058] High resolution: Visual sensors (such as cameras or depth cameras) can provide high-resolution images, enabling robots to more accurately determine details such as distance, size, and shape between objects, thus helping to perform fine operations.
[0059] 2. Autonomous navigation and positioning:
[0060] Visual SLAM (Simultaneous Localization and Mapping): Visual sensors enable visual SLAM technology, helping robots achieve autonomous localization and map building in unknown environments. Compared to traditional LiDAR-based SLAM, visual SLAM offers advantages in cost and adaptability, especially in environments without LiDAR.
[0061] Environmental recognition and path planning: The robot can identify obstacles and target objects in the environment in real time through visual perception, and combine them with path planning algorithms to formulate a more intelligent movement trajectory, thereby effectively avoiding collisions.
[0062] 3. Enhanced object recognition and classification capabilities:
[0063] Visual recognition technology: Visual sensors can use image processing and deep learning algorithms to identify and classify various objects. For example, in industrial applications, robots can use visual sensors to identify parts of different shapes and colors and perform automated assembly, handling, and other tasks.
[0064] Adaptability in complex environments: Visual sensors can handle more diverse environments, such as high-contrast, complex backgrounds, or low-light environments, thereby improving the robot's adaptability in complex situations.
[0065] Figure 1 This is a schematic diagram illustrating the workflow of an obstacle avoidance system for a robot, provided as an embodiment of this disclosure. Figure 1 As shown, it includes:
[0066] Step 1, Sensor Data: Data is collected through image acquisition chips and radar sensors.
[0067] Step 2, Visual Odometry: Pose estimation based on visual input.
[0068] Step 3, Backend Optimization: Integrate camera pose and loop closure detection information measured by visual odometry at different times, and perform global optimization to obtain consistent trajectories and maps;
[0069] Step 4, Loop Closing: During map building, visual sensors are used to determine whether the robot has returned to the same historical location.
[0070] Step 5: Mapping: Based on the estimated trajectory, create a map that meets the task requirements.
[0071] Figure 2 This is a flowchart illustrating an obstacle avoidance method for a robot provided in an embodiment of this disclosure. Figure 2 As shown, it includes:
[0072] Step 201: Acquire image data within the target area collected by the acquisition device, wherein the image data carries location information;
[0073] In this step, the image acquisition module continuously captures camera images through the input device.
[0074] Figure 3 This is a flowchart illustrating the data acquisition process in this embodiment.
[0075] Step 202: Determine the operating status of the target robot within the target area based on the image data and the location information;
[0076] In one embodiment, determining the operating status of the target robot within the target area based on the image data and the location information includes:
[0077] If every frame in the image data is identical, and / or the position information carried by each frame is identical, then the target robot is determined to be malfunctioning.
[0078] In this embodiment, by analyzing whether each frame is the same, the robot's position is determined by combining RTK / integrated navigation, and the robot's angle and orientation are determined by the IMU / encoder module to determine whether the robot is in motion or stuck in a circling state.
[0079] Step 203: Determine whether there are obstacles in the target area based on the image data and the operating status of the target robot in the target area;
[0080] In one embodiment, such as Figure 4 As shown, determining whether there are obstacles in the target area based on the image data and the target robot's operating status within the target area includes:
[0081] Step 401: Extract key image frames from the image data and extract features from the key image frames;
[0082] Step 402: Detect and classify obstacles using a deep learning model and the features of the key image frames;
[0083] Step 403: Determine whether there are obstacles in the target area based on the detection and classification results and the operating status of the target robot in the target area.
[0084] In this embodiment, after filtering a captured frame, keyframes are extracted, features are extracted from the image, objects are detected and classified using a deep learning model, and spatial position estimation is combined to determine whether there are obstacles.
[0085] Step 203: If there are obstacles in the target area, mark the target area as a prohibited area and replan the passable area.
[0086] Optionally, such as Figure 5 As shown, the method further includes:
[0087] Step 501: If there are obstacles in the target area, mark the obstacles, locate and filter them out;
[0088] Step 502: Send the image containing the obstacle to the user terminal bound to the target robot;
[0089] Step 503: Instruct the user terminal to receive and display an image containing the obstacle;
[0090] Step 504: Receive the return information sent by the user terminal, the return information being used to indicate that the corresponding obstacle in the image containing the obstacle is identified as an obstacle or a hazard;
[0091] Step 505: Mark the area where the obstacle is located as a prohibited area based on the obstacle information.
[0092] In this embodiment, obstacles are located and filtered after being marked, and then transmitted to the user's mobile APP bound to the robot via wireless Wi-Fi network. After the user views the image sent to the mobile APP and confirms that it is an obstacle or danger, the robot automatically marks the area as a prohibited area based on the size, distance and location of the obstacle.
[0093] Optionally, the method further includes:
[0094] The target robot's mission trajectory is constructed based on the passable area and the prohibited area.
[0095] A task requirement map for the target robot is created based on the target robot's task trajectory and the image data.
[0096] In this embodiment, the robot re-detects objects based on the depth model and, combined with spatial position estimation, re-plans the passable area.
[0097] Figure 6 This is a flowchart illustrating a robot obstacle avoidance method provided in an embodiment of the present disclosure.
[0098] Figure 7 This is a structural diagram of an obstacle avoidance device for a robot provided in an embodiment of this disclosure. Figure 7 As shown, the obstacle avoidance device of the robot includes: an acquisition module 701, a judgment module 702, a determination module 703, and a marking module 704; wherein, the acquisition module 701 is used to acquire image data within a target area collected by the acquisition device, and the image data carries location information; the judgment module 702 determines the running status of the target robot within the target area based on the image data and the location information; the determination module 703 determines whether there are obstacles in the target area based on the image data and the running status of the target robot within the target area; the marking module 704 is used to mark the target area as a prohibited running area and re-plan a passable area when there are obstacles in the target area.
[0099] In one embodiment, the determining module is specifically used for:
[0100] If every frame in the image data is identical, and / or the position information carried by each frame is identical, the target robot is determined to be malfunctioning.
[0101] The determining module is specifically used for:
[0102] Extract key image frames from the image data, and extract features from the key image frames;
[0103] Obstacles are detected and classified using a deep learning model and the features of the key image frames;
[0104] Based on the detection and classification results and the operating status of the target robot within the target area, it is determined whether there are obstacles in the target area.
[0105] Figure 8 This is a schematic diagram of the robot in this embodiment.
[0106] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 9 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method for implementing a VR game based on emotion analysis. It includes: memory and a processor; the memory stores the computer program; and the processor executes the computer program to implement any step in the above-described method for implementing a VR game based on emotion analysis.
[0107] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can perform any of the steps in the above-described method for implementing a VR game based on emotion analysis.
[0108] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0109] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0112] Although preferred embodiments of this disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this disclosure.
[0113] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.
Claims
1. An obstacle avoidance method for a robot, characterized in that, The obstacle avoidance method includes: Acquire image data within a target area collected by the acquisition device, wherein the image data carries location information; The operating status of the target robot within the target area is determined based on the image data and the location information; The presence of obstacles in the target area is determined based on the image data and the operating status of the target robot within the target area. If there are obstacles in the target area, the target area will be marked as a prohibited area, and a passable area will be replanned.
2. The method according to claim 1, characterized in that, The step of determining the operating status of the target robot within the target area based on the image data and the location information includes: If every frame in the image data is identical, and / or the position information carried by each frame is identical, then the target robot is determined to be malfunctioning.
3. The method according to claim 2, characterized in that, The step of determining whether there are obstacles in the target area based on the image data and the operating status of the target robot in the target area includes: Extract key image frames from the image data, and extract features from the key image frames; Obstacles are detected and classified using a deep learning model and the features of the key image frames; Based on the detection and classification results and the operating status of the target robot within the target area, it is determined whether there are obstacles in the target area.
4. The method according to claim 3, characterized in that, The method further includes: If obstacles exist in the target area, the obstacles are marked, and the obstacles are located and filtered out. The image containing the obstacle is sent to the user terminal that is bound to the target robot. The user terminal is instructed to receive and display an image containing the obstacle; The system receives return information sent by the user terminal, the return information being used to indicate that the corresponding obstacle in the image containing the obstacle is identified as an obstacle or a hazard; The area where the obstacle is located is marked as a no-entry zone based on the obstacle information.
5. The method according to claim 4, characterized in that, The information about the obstacle includes the size of the obstacle, the location of the obstacle, and the distance between the target robot and the obstacle.
6. The method according to claim 1, characterized in that, The method further includes: The target robot's mission trajectory is constructed based on the passable area and the prohibited area. A task requirement map for the target robot is created based on the target robot's task trajectory and the image data.
7. An obstacle avoidance device for a robot, characterized in that, The robot's obstacle avoidance device includes: an acquisition module, a judgment module, a determination module, and a marking module; wherein, The acquisition module is used to acquire image data within the target area collected by the acquisition device, and the image data carries location information; The judgment module determines the operating status of the target robot within the target area based on the image data and the location information. The determining module determines whether there are obstacles in the target area based on the image data and the operating status of the target robot in the target area; The marking module is used to mark the target area as a prohibited area when there are obstacles in the target area, and to re-plan the passable area.
8. The apparatus according to claim 7, characterized in that, The judgment module is specifically used for: If every frame in the image data is identical, and / or the position information carried by each frame is identical, the target robot is determined to be malfunctioning. The determining module is specifically used for: Extract key image frames from the image data, and extract features from the key image frames; Obstacles are detected and classified using a deep learning model and the features of the key image frames; Based on the detection and classification results and the operating status of the target robot within the target area, it is determined whether there are obstacles in the target area.
9. A computer device, comprising: A memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.