A method and system for avoiding obstacles in motion of a sweeping robot

Abstract

The present application relates to the field of smart home technology, and in particular to a method and system for avoiding obstacles in the motion of a sweeping robot. The method for avoiding obstacles in the motion of a sweeping robot is applied to a sweeping robot, which includes a radar scanning device, a camera device, and a control chip. The method is applied to the control chip, and the method includes: obtaining the scanning results of the radar scanning device and the on-site images taken by the camera device; analyzing the scanning results and the on-site images to determine the type of obstacles in the area to be cleaned; and determining whether there are uncleanable obstacles based on the type of obstacles. Based on the type and characteristics of the obstacle, the feasibility of the sweeping robot to clean this type of obstacle is evaluated to avoid unnecessary cleaning attempts and damage to the obstacle. By predicting the motion trajectory of dynamic obstacles, the obstacle avoidance path is planned in advance to ensure that the sweeping robot can safely avoid dynamic obstacles.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more particularly to a method and system for avoiding obstacles while moving a sweeping robot. Background Art

[0002] With the popularity of smart home devices, sweeping robots have become the core tool for daily cleaning in modern families. In order to improve the cleaning efficiency of sweeping robots, in addition to strong suction and intelligent path planning capabilities, obstacle avoidance technology is also one of its most important functions.

[0003] Most existing sweeping robots use simple collision detection and collision avoidance strategies, relying on sensors (such as ultrasonic, infrared sensors, collision switches, etc.) to detect surrounding obstacles and avoid collisions by reverse movement or steering. However, these methods are often limited to simple physical obstacle avoidance and cannot effectively cope with complex and changeable actual cleaning environments. Therefore, how to effectively avoid obstacles in various obstacles and areas with high cleaning difficulty has become one of the key technologies to improve the functions of sweeping robots. Summary of the invention

[0004] The present application provides a method and system for avoiding obstacles in motion of a sweeping robot to solve the above-mentioned problems.

[0005] In a first aspect, the present application provides a method for avoiding obstacles while moving a sweeping robot, the method for avoiding obstacles while moving a sweeping robot is applied to the sweeping robot, the sweeping robot includes a radar scanning device, a camera device, and a control chip, the method is applied to the control chip, and the method includes:

[0006] Obtaining the scanning results of the radar scanning device and the on-site images taken by the camera device; analyzing the scanning results and the on-site images to determine the types of obstacles in the area to be cleaned;

[0007] Determine whether there is an uncleanable obstacle based on the obstacle type;

[0008] If there is an uncleanable obstacle, judging whether there is interference between other obstacles and the uncleanable obstacle according to the scene image; if there is interference, determining the influence range of each uncleanable obstacle according to the obstacle type and the scene image;

[0009] Determine interference conditions between interfering obstacles according to the impact range;

[0010] According to the interference situation, the cleaning feasibility of the overlapping area of ​​each impact range is judged, and avoidance is performed based on the cleaning feasibility and the impact range.

[0011] Through this solution, the three-dimensional spatial information and on-site image information of the environment are obtained synchronously through radar scanning equipment and camera equipment, providing a data basis for subsequent obstacle identification and analysis. Through filtering and image preprocessing, the accuracy and clarity of the data are improved, providing high-quality input data for subsequent image analysis and three-dimensional reconstruction. Through image processing technology, the outline and shape of the obstacle are identified to provide a basis for the classification of obstacle types. By combining the radar scanning results and camera images, obstacles are classified, such as liquids, solids, pet hair, etc., to provide a basis for cleaning decisions. According to the type and characteristics of the obstacle, the feasibility of the sweeping robot for the type of obstacle is evaluated to avoid unnecessary cleaning attempts and damage to the obstacle. By predicting the motion trajectory of dynamic obstacles, the obstacle avoidance path is planned in advance to ensure that the sweeping robot can safely avoid dynamic obstacles. By fusing radar scanning data and camera image data, the accuracy and reliability of obstacle identification are improved, providing more comprehensive information support for cleaning strategies. According to the obstacle type, cleaning feasibility evaluation and dynamic obstacle avoidance strategy, the best cleaning path is planned to improve cleaning efficiency and safety.

[0012] Optionally, analyzing the scanning result and the on-site image to determine the type of obstacles in the area to be cleaned includes:

[0013] Analyzing the scanning results to determine the basic obstacle type;

[0014] Analyzing the scene image to determine the state of obstacles;

[0015] Determining the difficulty of clearing the obstacle according to the obstacle state;

[0016] The obstacle type is determined according to the obstacle clearing difficulty and the basic obstacle type.

[0017] Through this solution, the sweeping robot can preliminarily determine the basic type of obstacles, such as solid, liquid or void, through the radar scanning results, which helps the robot to preliminarily plan the cleaning path and obstacle avoidance strategy. Through the on-site images obtained by the camera device, the sweeping robot can further identify the specific state of the obstacle, such as static or dynamic, and the specific characteristics of the obstacle, such as color, shape, texture, etc., which helps the robot to more accurately identify the type of obstacle. By analyzing the state and characteristics of the obstacle, the sweeping robot can evaluate the cleaning difficulty, such as whether it is easy to clean, whether special treatment is required, etc., which helps the robot decide whether to clean the obstacle or avoid it. Combining the scanning results, on-site images, obstacle status and cleaning difficulty assessment, the sweeping robot can finally determine the type of obstacle, which helps the robot make the most appropriate cleaning decision, such as cleaning, avoiding or pushing the obstacle.

[0018] Optionally, the basic obstacle type includes a liquid type, and determining the obstacle clearing difficulty according to the obstacle state includes:

[0019] Analyzing the scene image to determine obstacle features;

[0020] Determining the color of the liquid according to the obstacle characteristics;

[0021] Retrieving a preset liquid stain database, and determining the liquid type according to the liquid color and the preset liquid stain database;

[0022] determining whether liquid deposition exists according to the color of the liquid;

[0023] If it is determined that liquid deposition is present, determining the floor adsorption level based on the type of liquid;

[0024] The deposition state is determined according to the color of the liquid, and the difficulty of clearing the obstacle is determined according to the floor adsorption level and the deposition state.

[0025] Through this solution, the visual features of obstacles are extracted through image processing technology, which helps the sweeping robot distinguish different types of obstacles, especially liquids and other solid obstacles. By identifying the color characteristics of liquid obstacles, the sweeping robot can distinguish different types of liquid stains, providing a basis for subsequent liquid type identification. By comparing the preset liquid stain database, the sweeping robot can accurately identify the specific type of liquid obstacles, which helps the robot choose the appropriate cleaning strategy. By analyzing the color and distribution of the liquid, the sweeping robot can determine whether the liquid has penetrated or deposited on the floor. According to the type of liquid and the material of the floor, the sweeping robot can evaluate the degree of adsorption of the liquid on the floor, which helps the robot adjust the cleaning intensity and number of sweeping times. By integrating information on liquid color, deposition state and floor adsorption level, the sweeping robot can evaluate the difficulty of cleaning liquid obstacles and adjust the cleaning strategy accordingly.

[0026] Optionally, determining the influence range of each uncleanable obstacle according to the obstacle type and the scene image includes:

[0027] Obtaining the cleaning range of the cleaning robot per unit time;

[0028] Determining an extension area according to the obstacle type and the on-site image;

[0029] An impact range is determined according to the cleaning range and the extended area.

[0030] Through this solution, by analyzing the robot's cleaning path and cleaning mode, the area that the robot can clean in a unit time is determined, which helps the robot plan the cleaning route and avoid repeated cleaning or missing areas. Through on-site image analysis, the specific location and characteristics of obstacles are identified. According to the type of obstacle (such as liquid, solid, etc.), the surrounding area that may be affected by the obstacle, that is, the extended area, is determined. For example, liquid obstacles may spread, so their extended area may be larger than that of solid obstacles. Provide additional cleaning instructions to the robot, such as bypassing obstacles or focusing on cleaning specific areas. Combine the cleaning range and extended area to evaluate which areas need to be focused on and which areas need to be avoided. According to the dynamic changes of obstacles (such as moving pets, children, etc.), the impact range is dynamically adjusted to ensure that the robot can effectively avoid dynamic obstacles during the cleaning process. Optimize the robot's cleaning path to ensure that all areas that need to be cleaned can be covered during the cleaning process while avoiding damage to obstacles.

[0031] Optionally, the determining the extension area according to the obstacle type and the scene image includes:

[0032] Determining whether the obstacle is in motion according to the scene image;

[0033] If the object is in motion, the position of the obstacle in the scene image is analyzed according to the shooting time of the scene image;

[0034] Determine the movement direction and angle of the obstacle according to the position;

[0035] Determining a final position according to the movement direction and the movement angle;

[0036] According to the final position, an extension area is determined.

[0037] Through this solution, by analyzing continuous image frames, the robot can identify whether the obstacle is stationary or moving, so that the robot can adjust the cleaning path in real time to avoid moving obstacles. Using the timestamp information of the image, the robot can track the position of the obstacle at different time points. By comparing the position changes of the obstacle in continuous frames, the robot can calculate the movement direction and angle of the obstacle. It helps the robot predict the future position of the obstacle and plan the cleaning path accordingly. Through the prediction algorithm, the robot can estimate the final position of the obstacle in the next cleaning cycle. It provides enough time for the robot to adjust the cleaning strategy to avoid collision with the obstacle. According to the final position and movement trend of the obstacle, the robot can determine the area that needs extra attention, that is, the extension area. It helps the robot avoid cleaning obstacles, while ensuring the integrity and safety of the cleaning process and preventing the sweeping robot from being entangled in the hair on the surface of the animal.

[0038] Optionally, determining the extension area according to the final position includes:

[0039] Analyze the scene image to determine whether there is any garbage left in the obstacle;

[0040] If it exists, the type and scope of garbage retention are determined based on the image analysis results;

[0041] An extension area is determined according to the garbage retention type, the garbage retention range and the final location.

[0042] With this solution, the robot can identify whether there is garbage left on obstacles through image analysis technology, which is very important to avoid contamination and damage during cleaning. The robot can distinguish different types of garbage, such as food scraps, paper, plastic, etc., which helps to choose the appropriate cleaning method. The robot can accurately locate the area where garbage is left, providing a basis for cleaning path planning. The robot can plan the extended area that needs to be cleaned according to the type and distribution of garbage to ensure comprehensive cleaning. The robot can avoid cleaning obstacles near the garbage, reducing the risk of collision and damage. If the garbage is dynamic, the robot can dynamically adjust the cleaning path according to its final position to avoid cleaning the garbage.

[0043] Optionally, determining the influence range of each uncleanable obstacle according to the obstacle type and the scene image includes:

[0044] Determining whether the uncleanable obstacle is movable according to the obstacle type;

[0045] If the mobility exists, determining the obstacle position of the uncleanable obstacle according to the on-site image;

[0046] According to the obstacle position, planning a convergence path;

[0047] According to the gathering path, the uncleanable obstacles are gathered and a gathering image is obtained;

[0048] The grouping image is analyzed to determine a grouping range, and the grouping range is determined as an influence range.

[0049] This solution helps the robot distinguish which obstacles are movable and which are immovable. For movable obstacles, the robot can try to gather them, while for immovable obstacles, the robot needs to plan a detour. Through image processing technology, the robot can accurately locate the position of obstacles, which is crucial for subsequent path planning and gathering operations. Plan an optimal path from the robot's current position to the obstacle to ensure that the robot can approach and handle the obstacle efficiently. The robot moves along the planned path, gathers the obstacles, and obtains images of the gathering process through the camera. These images are very important for subsequent analysis and decision-making. By analyzing the gathered images, the robot can determine the range of the obstacles after they are gathered. This range will become the area that the robot needs to pay special attention to during the cleaning process.

[0050] Optionally, planning a convergence path according to the obstacle position includes:

[0051] Determining the infrastructure of the uncleanable obstacle based on the obstacle type;

[0052] Determining the cleaning characteristics of the uncleanable obstacle based on the infrastructure;

[0053] Determine the priority cleaning nodes according to the cleaning characteristics;

[0054] According to the priority cleaning nodes, a gathering path is planned.

[0055] Through this solution, by analyzing the type of obstacle, the robot can identify the physical characteristics of the obstacle and determine its basic structure, which helps the robot understand the stability and cleaning difficulty of the obstacle. After understanding the basic structure of the obstacle, the robot can determine its cleaning characteristics. Based on the cleaning characteristics, the robot can determine which areas or parts need to be cleaned first, which helps the robot optimize the cleaning order and improve cleaning efficiency. Planning the optimal path from the current position to the priority cleaning node helps the robot reach the cleaning node quickly and safely. When planning the path, the robot needs to avoid obstacles, which helps the robot avoid collisions with obstacles during the cleaning process and protect the robot itself and the obstacles from damage.

[0056] Optionally, determining whether there is an uncleanable obstacle according to the obstacle type includes:

[0057] Determining the height of the obstacle according to the scene image;

[0058] Obtaining the cleaning properties of the cleaning robot;

[0059] Determining an obstacle crossing level of the cleaning robot according to the cleaning attributes;

[0060] According to the obstacle crossing level and the obstacle height, it is determined whether the obstacle is an uncleanable obstacle.

[0061] Through this solution, by analyzing the scene image, the robot can obtain three-dimensional information of obstacles, which helps the robot evaluate whether it can overcome the obstacle and how to adjust the cleaning strategy. Understanding the robot's cleaning properties helps determine its cleaning capabilities and limitations, so that it can make reasonable decisions when encountering obstacles. Determining the obstacle crossing level enables the robot to judge whether it can safely overcome the obstacle or whether it needs to adopt other cleaning strategies. Helping the robot avoid trying to clean obstacles that it cannot handle, thereby protecting the robot and the obstacles from damage.

[0062] In a second aspect, the present application provides a robot vacuum cleaner motion obstacle avoidance system, the system comprising:

[0063] A type analysis module is used to obtain the scanning results of the radar scanning device and the on-site images taken by the camera device; analyze the scanning results and the on-site images to determine the type of obstacles in the area to be cleaned;

[0064] An obstacle analysis module is used to determine whether there is an uncleanable obstacle according to the obstacle type;

[0065] An influence range determination module, for determining, if an uncleanable obstacle exists, whether interference exists between other obstacles and the uncleanable obstacle according to the scene image, and if interference exists, determining the influence range of each uncleanable obstacle according to the obstacle type and the scene image;

[0066] An interference situation determination module, used to determine the interference situation between the interfering obstacles according to the influence range;

[0067] The avoidance range determination module is used to determine the feasibility of cleaning the overlapping areas of each impact range according to the interference situation, and to avoid according to the cleaning feasibility and the impact range.

[0068] Optionally, when the type analysis module analyzes the scanning result and the on-site image to determine the type of obstacles in the area to be cleaned, it is used to:

[0069] Analyzing the scanning results to determine the basic obstacle type;

[0070] Analyzing the scene image to determine the state of obstacles;

[0071] Determining the difficulty of clearing the obstacle according to the obstacle state;

[0072] The obstacle type is determined according to the obstacle clearing difficulty and the basic obstacle type.

[0073] Optionally, the basic obstacle type includes a liquid type, and the type analysis module determines the difficulty of obstacle cleaning according to the obstacle state, and is used to:

[0074] Analyzing the scene image to determine obstacle features;

[0075] Determining the color of the liquid according to the obstacle characteristics;

[0076] Retrieving a preset liquid stain database, and determining the liquid type according to the liquid color and the preset liquid stain database;

[0077] determining whether liquid deposition exists according to the color of the liquid;

[0078] If it is determined that liquid deposition is present, determining the floor adsorption level based on the type of liquid;

[0079] The deposition state is determined according to the color of the liquid, and the difficulty of clearing the obstacle is determined according to the floor adsorption level and the deposition state.

[0080] Optionally, when the influence range determination module determines the influence range of each uncleanable obstacle according to the obstacle type and the scene image, it is used to:

[0081] Obtaining the cleaning range of the cleaning robot per unit time;

[0082] Determining an extension area according to the obstacle type and the on-site image;

[0083] An impact range is determined according to the cleaning range and the extended area.

[0084] Optionally, when the impact range determination module determines the extension area according to the obstacle type and the scene image, it is used to:

[0085] Determining whether the obstacle is in motion according to the scene image;

[0086] If the object is in motion, the position of the obstacle in the scene image is analyzed according to the shooting time of the scene image;

[0087] Determine the movement direction and angle of the obstacle according to the position;

[0088] Determining a final position according to the movement direction and the movement angle;

[0089] According to the final position, an extension area is determined.

[0090] Optionally, when the impact range determination module determines the extension area according to the final position, it is used to:

[0091] Analyze the scene image to determine whether there is any garbage left in the obstacle;

[0092] If it exists, the type and scope of garbage retention are determined based on the image analysis results;

[0093] An extension area is determined according to the garbage retention type, the garbage retention range and the final location.

[0094] Optionally, when the influence range determination module determines the influence range of each uncleanable obstacle according to the obstacle type and the scene image, it is used to:

[0095] Determining whether the uncleanable obstacle is movable according to the obstacle type;

[0096] If the mobility exists, determining the obstacle position of the uncleanable obstacle according to the on-site image;

[0097] According to the obstacle position, planning a convergence path;

[0098] According to the gathering path, the uncleanable obstacles are gathered and a gathering image is obtained;

[0099] The grouping image is analyzed to determine a grouping range, and the grouping range is determined as an influence range.

[0100] Optionally, when the impact range determination module plans the convergence path according to the obstacle position, it is used to:

[0101] Determining the infrastructure of the uncleanable obstacle based on the obstacle type;

[0102] Determining the cleaning characteristics of the uncleanable obstacle based on the infrastructure;

[0103] Determine the priority cleaning nodes according to the cleaning characteristics;

[0104] According to the priority cleaning nodes, a gathering path is planned.

[0105] Optionally, when the obstacle analysis module determines whether there is an uncleanable obstacle according to the obstacle type, it is used to:

[0106] Determining the height of the obstacle according to the scene image;

[0107] Obtaining the cleaning properties of the cleaning robot;

[0108] Determining an obstacle crossing level of the cleaning robot according to the cleaning attributes;

[0109] According to the obstacle crossing level and the obstacle height, it is determined whether the obstacle is an uncleanable obstacle. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0111] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0112] Figure 2 A flowchart of a sweeping robot motion obstacle avoidance method provided in one embodiment of the present application;

[0113] Figure 3 A schematic diagram of the structure of a sweeping robot motion obstacle avoidance system provided in one embodiment of the present application. DETAILED DESCRIPTION

[0114] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0115] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0116] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0117] With the popularity of smart home devices, sweeping robots have become the core tool for daily cleaning in modern families. In order to improve the cleaning efficiency of sweeping robots, in addition to strong suction and intelligent path planning capabilities, obstacle avoidance technology is also one of its most important functions.

[0118] Most existing sweeping robots use simple collision detection and collision avoidance strategies, relying on sensors (such as ultrasonic, infrared sensors, collision switches, etc.) to detect surrounding obstacles and avoid collisions by reverse movement or steering. However, these methods are often limited to simple physical obstacle avoidance and cannot effectively cope with complex and changeable actual cleaning environments. Therefore, how to effectively avoid obstacles in various obstacles and areas with high cleaning difficulty has become one of the key technologies to improve the functions of sweeping robots.

[0119] Based on this, the present application provides a method and system for avoiding obstacles in the motion of a sweeping robot, which obtains the scanning results of a radar scanning device and the on-site images taken by a camera device; analyzes the scanning results and the on-site images to determine the type of obstacles in the area to be cleaned; determines whether there are uncleanable obstacles according to the type of obstacles; if there are obstacles, determines the impact range according to the obstacle type and the on-site images, and avoids them according to the impact range. Through the radar scanning device and the camera device, the three-dimensional spatial information and on-site image information of the environment are synchronously obtained to provide a data basis for subsequent obstacle identification and analysis. Through filtering and image preprocessing, the accuracy and clarity of the data are improved, and high-quality input data is provided for subsequent image analysis and three-dimensional reconstruction. Through image processing technology, the outline and shape of the obstacle are identified to provide a basis for the classification of the obstacle type. Combined with the radar scanning results and the camera images, the obstacles are classified, such as liquids, solids, pet hair, etc., to provide a basis for cleaning decisions. According to the type and characteristics of the obstacle, the feasibility of the sweeping robot to clean the obstacle of this type is evaluated to avoid unnecessary cleaning attempts and damage to the obstacle. By predicting the motion trajectory of dynamic obstacles, the obstacle avoidance path is planned in advance to ensure that the robot vacuum cleaner can safely avoid dynamic obstacles. By integrating radar scanning data and camera image data, the accuracy and reliability of obstacle recognition are improved, providing more comprehensive information support for cleaning strategies. According to the obstacle type, cleaning feasibility assessment and dynamic obstacle avoidance strategy, the best cleaning path is planned to improve cleaning efficiency and safety.

[0120] Figure 1A schematic diagram of an application scenario provided by the present application is used when the sweeping robot avoids obstacles. Specifically, the method provided by the present application is applied to the control chip of any sweeping robot, and the control chip interacts with the radar scanning device and the camera device in the sweeping robot 001. The three-dimensional spatial information and on-site image information of the environment are synchronously acquired through the radar scanning device and the camera device carried by the sweeping robot, providing a data basis for subsequent obstacle identification and analysis. Through filtering and image preprocessing, the accuracy and clarity of the data are improved, and high-quality input data is provided for subsequent image analysis and three-dimensional reconstruction. Through image processing technology, the outline and shape of the obstacle are identified to provide a basis for the classification of obstacle types. Combined with the radar scanning results and the camera images, the obstacles are classified, such as liquids, solids, pet hair, etc., to provide a basis for cleaning decisions. According to the type and characteristics of the obstacle, the feasibility of the sweeping robot for cleaning this type of obstacle is evaluated to avoid unnecessary cleaning attempts and damage to the obstacle. By predicting the motion trajectory of dynamic obstacles, the obstacle avoidance path is planned in advance to ensure that the sweeping robot can safely avoid dynamic obstacles. By fusing radar scanning data and camera image data, the accuracy and reliability of obstacle recognition are improved, providing more comprehensive information support for cleaning strategies. According to the obstacle type, cleaning feasibility assessment and dynamic obstacle avoidance strategy, the best cleaning path is planned to improve cleaning efficiency and safety. For specific implementation methods, please refer to the following embodiments.

[0121] Figure 2 This is a flow chart of a sweeping robot motion obstacle avoidance method provided by an embodiment of the present application. The method of this embodiment can be applied to the control chip in the above scenario. Figure 2 As shown, the method includes:

[0122] S201, obtaining the scanning results of the radar scanning device and the on-site images taken by the camera device; analyzing the scanning results and the on-site images to determine the types of obstacles in the area to be cleaned;

[0123] The scanning result may be point cloud data obtained after the radar scanning device scans the surrounding environment. These data usually include obstacle information in the environment, such as the location, distance, size, etc. of the obstacle.

[0124] The on-site image can be a real-time image of the current cleaning area taken by a camera device. These images contain rich visual information, such as the shape, color, texture, etc. of the obstacles.

[0125] The area to be cleaned may be an area that the robot needs to clean, and may be an entire room, a specific room, a part of a room, or a user-specified cleaning area.

[0126] The obstacle type can be various types of objects in the environment that may affect the cleaning of the sweeping robot, such as liquids, solids, pets, and movable objects.

[0127] Specifically, the radar scanning device and camera device on the sweeping robot are started to synchronously obtain the environmental scanning results and the on-site images. The radar scanning results are filtered to remove noise and interference and extract effective obstacle information. The on-site images taken by the camera are preprocessed, including denoising and contrast enhancement, to facilitate subsequent image analysis. Image processing techniques such as edge detection and feature extraction are used to identify the outline of obstacles from the on-site images. Combined with the radar scanning results, the obstacles are reconstructed in three dimensions to obtain information such as the location and size of the obstacles.

[0128] S202, determining whether there is an uncleanable obstacle according to the obstacle type;

[0129] Uncleanable obstacles can be obstacles that the robot vacuum cleaner cannot or should not clean during the cleaning process. These obstacles may be unable to be effectively cleaned by the robot vacuum cleaner due to their physical characteristics, location, size or other reasons, such as large furniture, valuables, immovable obstacles, etc.

[0130] Specifically, obstacles are classified into categories such as liquid, solid, pet hair, etc. according to their shape, color, texture and other features, combined with the preset obstacle database. According to the obstacle type, the feasibility of the robot vacuum cleaner to clean the obstacle of that type is evaluated. For obstacles that cannot be cleaned, such as large furniture and pets, their uncleanability is determined and avoidance paths are planned.

[0131] S203: If there is an uncleanable obstacle, determine whether there is interference between the remaining obstacles and the uncleanable obstacle based on the scene image; if there is interference, determine the influence range of each uncleanable obstacle based on the obstacle type and the scene image.

[0132] The impact range may be the area where obstacles affect the cleaning path and cleaning effect of the sweeping robot, such as static obstacles, dynamic obstacles, liquid stains, etc.

[0133] Specifically, if there are uncleanable obstacles, the relationship between any uncleanable obstacle and other surrounding obstacles is determined based on the on-site image. For example, if a liquid is judged to be uncleanable due to its viscosity and diffusion state, other obstacles above the liquid can be cleaned independently, but at this time they should also be judged as uncleanable. Based on the current impact situation, the uncleanable range is superimposed to determine the impact range. For example, for liquids, the impact range may be the area of ​​the floor covered by the liquid and other surrounding obstacles. Fuse the radar scanning data and camera image data to obtain more comprehensive and accurate obstacle information. Improve the accuracy and reliability of obstacle identification through data fusion.

[0134] S204: Determine the interference situation between the interfering obstacles according to the impact range.

[0135] Specifically, according to the impact range, determine the specific interference between any uncleanable obstacle and other surrounding obstacles, such as the distance between obstacles (whether the robot vacuum cleaner can pass through) or whether the uncleanable obstacle will be touched during the cleaning process.

[0136] S205: According to the interference situation, the feasibility of cleaning the overlapping areas of each impact range is judged, and avoidance is performed according to the cleaning feasibility and the impact range.

[0137] Specifically, according to the obstacle type, interference situation, and cleaning feasibility, the mutual interference between the impact ranges of each obstacle is evaluated, and the range that needs to be cleaned caused by the current obstacle is judged, whether it can be cleaned, and the best cleaning path is planned; for example, if water and bottles are together, these two cannot be cleaned, because the bottle may be touched during the cleaning process, causing the water in the bottle to flow out further. At this time, an extended area should be set for the bottle body, and it cannot be touched during the cleaning process. Or pet hair and pet urine are stuck together. After the first identification of pet urine, the pet urine area is set for special cleaning (currently it will not be cleaned, but the area will be specially cleaned later). During the cleaning of the hair this time, pet urine will fall out around, so the hair should also be set in the extended area, and it cannot be touched during this cleaning process. For dynamic obstacles, such as moving pets and children, their future positions are predicted by analyzing their movement trajectory and speed. According to the predicted position, the cleaning path is dynamically adjusted to ensure that the sweeping robot can effectively avoid dynamic obstacles. During the cleaning process, the cleaning path is dynamically adjusted according to the real-time sensor data to ensure that the robot vacuum cleaner can avoid obstacles and complete the cleaning task. Through this solution, the three-dimensional spatial information and on-site image information of the environment are synchronously obtained through the radar scanning device and the camera device, providing a data basis for subsequent obstacle identification and analysis. Through filtering and image preprocessing, the accuracy and clarity of the data are improved, providing high-quality input data for subsequent image analysis and three-dimensional reconstruction. Through image processing technology, the outline and shape of the obstacle are identified to provide a basis for the classification of obstacle types. Combined with the radar scanning results and camera images, obstacles are classified, such as liquids, solids, pet hair, etc., to provide a basis for cleaning decisions. According to the type and characteristics of the obstacle, the feasibility of the robot vacuum cleaner to clean the obstacle of this type is evaluated to avoid unnecessary cleaning attempts and damage to the obstacle. By predicting the motion trajectory of dynamic obstacles, the obstacle avoidance path is planned in advance to ensure that the robot vacuum cleaner can safely avoid dynamic obstacles. By fusing radar scanning data and camera image data, the accuracy and reliability of obstacle identification are improved, providing more comprehensive information support for cleaning strategies. According to the obstacle type, cleaning feasibility assessment and dynamic obstacle avoidance strategy, the optimal cleaning path is planned to improve cleaning efficiency and safety.

[0138] In some embodiments, the scanning results and the on-site images are analyzed to determine the obstacle type in the area to be cleaned, including: analyzing the scanning results to determine the basic obstacle type; analyzing the on-site images to determine the obstacle status; determining the obstacle cleaning difficulty based on the obstacle status; determining the obstacle type based on the obstacle cleaning difficulty and the basic obstacle type.

[0139] The basic obstacle type may be the obstacle type preliminarily determined based on the radar scanning result, such as a solid obstacle, a liquid obstacle, a void obstacle, etc.

[0140] The obstacle state can be the dynamic characteristics of the obstacle, including whether it is in motion, the speed and direction of motion, etc.; static can be the obstacle being in a fixed state; dynamic can be the obstacle being in motion, such as a walking pet, a moving chair, etc.

[0141] The obstacle cleaning difficulty may be the difficulty level of the robot vacuum cleaner in clearing obstacles.

[0142] Specifically, radar scanning equipment is used to obtain three-dimensional spatial data of the environment, such as point cloud data or distance map. Camera equipment is used to obtain real-time images of the area to be cleaned, and image processing technology is used to analyze the state of obstacles, such as whether they are static or dynamic, and the movement trajectory of obstacles, by combining the scanning results and the on-site images. The type of obstacle is finally determined by combining the scanning results, on-site images, obstacle status, and cleaning difficulty assessment.

[0143] Through this solution, the sweeping robot can preliminarily determine the basic type of obstacles, such as solid, liquid or void, through the radar scanning results, which helps the robot to preliminarily plan the cleaning path and obstacle avoidance strategy. Through the on-site images obtained by the camera device, the sweeping robot can further identify the specific state of the obstacle, such as static or dynamic, and the specific characteristics of the obstacle, such as color, shape, texture, etc., which helps the robot to more accurately identify the type of obstacle. By analyzing the state and characteristics of the obstacle, the sweeping robot can evaluate the cleaning difficulty, such as whether it is easy to clean, whether special treatment is required, etc., which helps the robot decide whether to clean the obstacle or avoid it. Combining the scanning results, on-site images, obstacle status and cleaning difficulty assessment, the sweeping robot can finally determine the type of obstacle, which helps the robot make the most appropriate cleaning decision, such as cleaning, avoiding or pushing the obstacle.

[0144] In some embodiments, the scene image is analyzed to determine the characteristics of the obstacle; the color of the liquid is determined based on the characteristics of the obstacle; the preset liquid stain database is retrieved, and the liquid type is determined based on the liquid color and the preset liquid stain database; based on the liquid color, it is determined whether there is liquid deposition; if it is determined that liquid deposition exists, the floor adsorption level is determined based on the liquid type; based on the liquid color, the deposition state is determined, and the difficulty of clearing the obstacle is determined based on the floor adsorption level and the deposition state.

[0145] Obstacle features can be visual and spatial attributes of obstacles acquired by camera equipment, such as shape, color, size, position, etc.

[0146] Liquid color may be the visual color representation of a liquid stain.

[0147] The preset liquid stain database may be a database storing characteristics and properties of different liquid stains, and is used to help the sweeping robot identify and classify liquid stains.

[0148] Liquid type can be a specific kind of liquid barrier, classified according to its chemical and physical properties.

[0149] Liquid deposits may be deposits of liquid stains on floors or other surfaces.

[0150] Floor absorption level can be how well liquids absorb onto the floor, determining how easy or difficult it is to clean.

[0151] The deposition state can be the shape and distribution of the liquid deposited on the floor.

[0152] Specifically, a camera device is used to obtain on-site images, and the images are analyzed through image processing technology (such as color recognition and edge detection) to extract the features of obstacles. The color recognition function in the image processing technology is used to determine the color of the liquid obstacle. The identified liquid color is compared with the preset liquid stain database to determine the specific type of the liquid. By analyzing the depth and distribution of the liquid color, it is determined whether there is liquid deposition. According to the type of liquid, with reference to the known adsorption characteristics, the adsorption level of the liquid on the floor is evaluated. The difficulty of cleaning the liquid obstacle is evaluated by combining the liquid color, deposition state and floor adsorption level. For example, for liquid stains that are difficult to remove, the robot may choose to use a special cleaning mode or detergent.

[0153] Through this solution, the visual features of obstacles are extracted through image processing technology, which helps the sweeping robot distinguish different types of obstacles, especially liquids and other solid obstacles. By identifying the color characteristics of liquid obstacles, the sweeping robot can distinguish different types of liquid stains, providing a basis for subsequent liquid type identification. By comparing the preset liquid stain database, the sweeping robot can accurately identify the specific type of liquid obstacles, which helps the robot choose the appropriate cleaning strategy. By analyzing the color and distribution of the liquid, the sweeping robot can determine whether the liquid has penetrated or deposited on the floor. According to the type of liquid and the material of the floor, the sweeping robot can evaluate the degree of adsorption of the liquid on the floor, which helps the robot adjust the cleaning intensity and number of sweeping times. By integrating information on liquid color, deposition state and floor adsorption level, the sweeping robot can evaluate the difficulty of cleaning liquid obstacles and adjust the cleaning strategy accordingly.

[0154] In some embodiments, the cleaning range of the sweeping robot in unit time is obtained; the extended area is determined according to the obstacle type and the on-site image; and the affected range is determined according to the cleaning range and the extended area.

[0155] The cleaning range can be the area that the sweeping robot can clean within a unit of time, that is, the spatial coverage of the robot's cleaning activities.

[0156] The extended area can be an area near obstacles that may be affected by obstacles and need special attention, requiring the robot to pay extra attention or adjust the cleaning path.

[0157] Specifically, the area that the robot vacuum cleaner can clean in a unit of time is determined through sensor data (such as infrared, ultrasonic, laser radar, etc.) and cleaning path planning algorithms. Combined with the scanning results and on-site images, the characteristics of the obstacles (such as shape, size, location, etc.) are analyzed, and the extended area around the obstacles is determined, that is, the area where the robot vacuum cleaner may need to pay extra attention or adjust the cleaning path. For example, if a water bottle falls and water flows out, the specific conditions of the two obstacles, water and water bottle, need to be considered. If the water bottle is hit during the cleaning process, the water may continue to flow out. Or the spilled liquid is attached to pet hair. If the pet hair is cleaned, the area of ​​the liquid may be expanded. Comprehensively consider the cleaning range of the robot vacuum cleaner and the extended area around the obstacle to determine the final cleaning impact range, such as the main area that needs to be cleaned, the area near the obstacle that needs to be avoided, and the area that may be affected by the obstacle and needs special treatment.

[0158] Through this solution, by analyzing the robot's cleaning path and cleaning mode, the area that the robot can clean in a unit time is determined, which helps the robot plan the cleaning route and avoid repeated cleaning or missing areas. Through on-site image analysis, the specific location and characteristics of obstacles are identified. According to the type of obstacle (such as liquid, solid, etc.), the surrounding area that may be affected by the obstacle, that is, the extended area, is determined. For example, liquid obstacles may spread, so their extended area may be larger than that of solid obstacles. Provide additional cleaning instructions to the robot, such as bypassing obstacles or focusing on cleaning specific areas. Combine the cleaning range and extended area to evaluate which areas need to be focused on and which areas need to be avoided. According to the dynamic changes of obstacles (such as moving pets, children, etc.), the impact range is dynamically adjusted to ensure that the robot can effectively avoid dynamic obstacles during the cleaning process. Optimize the robot's cleaning path to ensure that all areas that need to be cleaned can be covered during the cleaning process while avoiding damage to obstacles.

[0159] In some embodiments, based on the scene image, it is determined whether the obstacle is in motion; if it is in motion, the position of the obstacle in the scene image is analyzed based on the shooting time of the scene image; based on the position, the movement direction and angle of the obstacle are determined; based on the movement direction and angle, the final position is determined; based on the final position, the extended area is determined.

[0160] The motion state may be a state of whether the position of the obstacle changes in space.

[0161] The location can be the spatial coordinates of the obstacle at a certain moment.

[0162] The movement direction may be the direction of the path along which the obstacle moves in space.

[0163] The movement angle can be the angle between the obstacle and the horizontal plane when it moves in space.

[0164] The final position may be the spatial coordinates that the obstacle is expected to reach after a period of movement.

[0165] Specifically, use image processing techniques, such as inter-frame difference or optical flow, to analyze continuous image frames to detect whether obstacles are moving. Use the timestamp information in the image and the robot's motion model to analyze the position of the obstacle in the image. Calculate the direction and angle of movement of the obstacle by comparing the position changes of the obstacle in consecutive frames. Use prediction algorithms, such as Kalman filtering or machine learning models, to predict the final position of the obstacle in the next cleaning cycle. Determine the area that needs additional attention, i.e., the extended area, based on the final position and movement trend of the obstacle.

[0166] Through this solution, by analyzing continuous image frames, the robot can identify whether the obstacle is stationary or moving, so that the robot can adjust the cleaning path in real time to avoid moving obstacles. Using the timestamp information of the image, the robot can track the position of the obstacle at different time points. By comparing the position changes of the obstacle in continuous frames, the robot can calculate the movement direction and angle of the obstacle. It helps the robot predict the future position of the obstacle and plan the cleaning path accordingly. Through the prediction algorithm, the robot can estimate the final position of the obstacle in the next cleaning cycle. It provides enough time for the robot to adjust the cleaning strategy to avoid collision with the obstacle. According to the final position and movement trend of the obstacle, the robot can determine the area that needs extra attention, that is, the extension area. It helps the robot avoid cleaning obstacles, while ensuring the integrity and safety of the cleaning process and preventing the sweeping robot from being entangled in the hair on the surface of the animal.

[0167] In some embodiments, the scene image is analyzed to determine whether there is garbage retained by the obstacle; if so, the type and range of the garbage retained are determined based on the image analysis results; and the extended area is determined based on the type of garbage retained, the range of garbage retained and the final location.

[0168] Garbage residue can be solid waste or liquid residue in the cleaning area of ​​the sweeping robot that is not cleaned up in time due to various reasons.

[0169] Waste retention types can be classified into categories of waste retention, distinguishing different types of solid waste or liquid residues.

[0170] The garbage retention range may be the size and distribution of the space occupied by the garbage retention.

[0171] Specifically, use image processing technology, such as edge detection and color recognition, to analyze on-site images to identify whether there is garbage left on obstacles, such as using color recognition technology to distinguish different types of garbage such as food residues and paper. According to the results of image processing, classify and identify the type of garbage left (such as food residues, paper, plastic, etc.), and use image segmentation technology to determine its range in the image to separate the garbage area. Analyze the impact of garbage retention on the surrounding environment, such as possible diffusion areas. If the garbage is dynamic, predict its possible movement trajectory, such as the diffusion of liquids. Combine the type, range and final location of garbage retention to plan extended areas that require extra attention.

[0172] With this solution, the robot can identify whether there is garbage left on obstacles through image analysis technology, which is very important to avoid contamination and damage during cleaning. The robot can distinguish different types of garbage, such as food scraps, paper, plastic, etc., which helps to choose the appropriate cleaning method. The robot can accurately locate the area where garbage is left, providing a basis for cleaning path planning. The robot can plan the extended area that needs to be cleaned according to the type and distribution of garbage to ensure comprehensive cleaning. The robot can avoid cleaning obstacles near the garbage, reducing the risk of collision and damage. If the garbage is dynamic, the robot can dynamically adjust the cleaning path according to its final position to avoid cleaning the garbage.

[0173] In some embodiments, based on the type of obstacle, it is determined whether the uncleanable obstacle is movable; if it is movable, the obstacle position of the uncleanable obstacle is determined based on the on-site image; based on the obstacle position, a gathering path is planned; based on the gathering path, the uncleanable obstacles are gathered and a gathering image is obtained; the gathering image is analyzed to determine the gathering range, and the gathering range is determined as the impact range.

[0174] Mobility can refer to whether an obstacle can be moved or relocated by the cleaning robot.

[0175] The obstacle position may be the specific coordinates or position of the obstacle in the image or real environment.

[0176] The convergence path can be the optimal path planning of the robot from the current position to the obstacle position.

[0177] The collapsing image may be an image captured by the robot during the process of collapsing obstacles.

[0178] Specifically, by analyzing the characteristics of the obstacle (such as shape, size, texture, etc.) and the on-site image, determine whether the obstacle is movable. Use image processing technology, such as edge detection, image segmentation, etc., to determine the specific location of the obstacle. Combined with the robot's motion model and cleaning strategy, use a path planning algorithm (such as the A* algorithm or the Dijkstra algorithm) to plan an optimal path to the obstacle. The robot moves along the planned path, gathers the obstacles, and obtains images of the gathering process through the camera. Analyze the gathered images, use image recognition technology to determine the boundaries of the obstacles, and then determine the gathering range, thereby determining the range of the obstacles after gathering, and determining the range of their impact on the cleaning process.

[0179] This solution helps the robot distinguish which obstacles are movable and which are immovable. For movable obstacles, the robot can try to gather them, while for immovable obstacles, the robot needs to plan a detour. Through image processing technology, the robot can accurately locate the position of obstacles, which is crucial for subsequent path planning and gathering operations. Plan an optimal path from the robot's current position to the obstacle to ensure that the robot can approach and handle the obstacle efficiently. The robot moves along the planned path, gathers the obstacles, and obtains images of the gathering process through the camera. These images are very important for subsequent analysis and decision-making. By analyzing the gathered images, the robot can determine the range of the obstacles after they are gathered. This range will become the area that the robot needs to pay special attention to during the cleaning process.

[0180] In some embodiments, a convergence path is planned based on the obstacle location, including: determining the infrastructure of the uncleanable obstacle based on the obstacle type; determining the cleaning characteristics of the uncleanable obstacle based on the infrastructure; determining the priority cleaning nodes based on the cleaning characteristics; and planning the convergence path based on the priority cleaning nodes.

[0181] Infrastructure can be the physical form and construction of the non-removable obstacle, such as its shape, size, material, and stability.

[0182] The cleaning characteristics may be the properties of the uncleanable obstacles that need to be considered during the cleaning process, such as whether they are fragile, whether they are easy to attract dust, whether specific tools are required, etc.

[0183] The priority cleaning node may be a specific area or part of an obstacle that the cleaning robot should first process during the cleaning process.

[0184] Specifically, through sensor data and image analysis, the types of obstacles are identified, such as fixed furniture, toys, electronic devices, etc., and their basic structures are determined. The basic structures of obstacles are analyzed to determine their cleaning characteristics, such as whether special care is needed to avoid damage, whether specific cleaning tools are needed, etc. For example, for fragile items, the robot needs to adopt a gentler cleaning method. According to the cleaning characteristics, determine which parts are the focus of cleaning and which parts need to be processed first. For example, for liquid stains, priority cleaning may be required to prevent spread; for food residues, priority cleaning may be required. Use path planning algorithms (such as A*, Dijkstra, RRT, etc.) to plan the optimal path from the robot's current position to the priority cleaning node. When planning the path, obstacles and the robot's own movement ability are considered to avoid obstacles to ensure that the path is safe and feasible.

[0185] Through this solution, by analyzing the type of obstacle, the robot can identify the physical characteristics of the obstacle and determine its basic structure, which helps the robot understand the stability and cleaning difficulty of the obstacle. After understanding the basic structure of the obstacle, the robot can determine its cleaning characteristics. Based on the cleaning characteristics, the robot can determine which areas or parts need to be cleaned first, which helps the robot optimize the cleaning order and improve cleaning efficiency. Planning the optimal path from the current position to the priority cleaning node helps the robot reach the cleaning node quickly and safely. When planning the path, the robot needs to avoid obstacles, which helps the robot avoid collisions with obstacles during the cleaning process and protect the robot itself and the obstacles from damage.

[0186] In some embodiments, the height of the obstacle is determined based on the scene image; the cleaning properties of the sweeping robot are obtained; the obstacle crossing level of the sweeping robot is determined based on the cleaning properties; and whether the obstacle is an uncleanable obstacle is determined based on the obstacle crossing level and the obstacle height.

[0187] The obstacle height may be the vertical dimension of the obstacle, ie, the distance from the bottom to the top.

[0188] Cleaning attributes may be the characteristics and capabilities of the sweeping robot during the cleaning process, such as the size of the cleaning tool, the cleaning mode, the suction strength, etc.

[0189] The obstacle crossing level may be the height of obstacles that the vacuum cleaner robot can safely cross.

[0190] Uncleanable obstacles may be obstacles that cannot be cleaned by the robot vacuum cleaner due to size, height or other characteristics.

[0191] Specifically, image processing technology (such as deep learning models, stereo vision, etc.) is used to analyze the scene image. The robot uses deep learning models or stereo vision technology to extract the height information of the obstacle from the image, and combines it with the known environmental map data to determine the actual height of the obstacle. The robot's cleaning properties are obtained through the robot's sensor data (such as wheel height, size of cleaning tools, etc.) and preset cleaning parameters (such as cleaning mode, cleaning intensity, etc.). The robot uses logical judgment or machine learning algorithms, combined with the robot's cleaning properties and the height of the obstacle, and uses algorithms (such as decision trees, neural networks, etc.) to determine the robot's obstacle crossing level. For example, if the height of the cleaning brush is less than the height of the obstacle, the robot's obstacle crossing level is low. Compare the robot's obstacle crossing level and the height of the obstacle. If the obstacle exceeds the robot's obstacle crossing capability, it is marked as an uncleanable obstacle.

[0192] Through this solution, by analyzing the scene image, the robot can obtain three-dimensional information of obstacles, which helps the robot evaluate whether it can overcome the obstacle and how to adjust the cleaning strategy. Understanding the robot's cleaning properties helps determine its cleaning capabilities and limitations, so that it can make reasonable decisions when encountering obstacles. Determining the obstacle crossing level enables the robot to judge whether it can safely overcome the obstacle or whether it needs to adopt other cleaning strategies. Helping the robot avoid trying to clean obstacles that it cannot handle, thereby protecting the robot and the obstacles from damage.

[0193] Figure 3 A schematic diagram of a sweeping robot motion obstacle avoidance system provided in one embodiment of the present application is shown in FIG. Figure 3 As shown, the sweeping robot motion obstacle avoidance system 300 of this embodiment includes: a type analysis module 301, an obstacle analysis module 302, an impact range determination module 303, an interference situation determination module 304, and an avoidance range determination module 305.

[0194] The type analysis module 301 is used to obtain the scanning results of the radar scanning device and the scene image taken by the camera device; analyze the scanning results and the scene image to determine the type of obstacles in the area to be cleaned;

[0195] The obstacle analysis module 302 is used to determine whether there is an uncleanable obstacle according to the obstacle type;

[0196] The influence range determination module 303 is used to determine whether there is interference between other obstacles and the uncleanable obstacle according to the scene image if there is an uncleanable obstacle, and if there is interference, determine the influence range of each uncleanable obstacle according to the obstacle type and the scene image;

[0197] An interference situation determination module 304 is used to determine the interference situation between the interfering obstacles according to the influence range;

[0198] The avoidance range determination module 305 is used to determine the feasibility of cleaning the overlapping area of ​​each impact range according to the interference situation, and to avoid according to the cleaning feasibility and the impact range.

[0199] Optionally, when the type analysis module 301 analyzes the scanning result and the on-site image to determine the type of obstacles in the area to be cleaned, it is used to:

[0200] Analyzing the scanning results to determine the basic obstacle type;

[0201] Analyzing the scene image to determine the state of obstacles;

[0202] Determining the difficulty of clearing the obstacle according to the obstacle state;

[0203] The obstacle type is determined according to the obstacle clearing difficulty and the basic obstacle type.

[0204] Optionally, the basic obstacle type includes a liquid type, and the type analysis module 301 determines the difficulty of obstacle cleaning according to the obstacle state, and is used to:

[0205] Analyzing the scene image to determine obstacle features;

[0206] Determining the color of the liquid according to the obstacle characteristics;

[0207] Retrieving a preset liquid stain database, and determining the liquid type according to the liquid color and the preset liquid stain database;

[0208] determining whether liquid deposition exists according to the color of the liquid;

[0209] If it is determined that liquid deposition is present, determining the floor adsorption level based on the type of liquid;

[0210] The deposition state is determined according to the color of the liquid, and the difficulty of clearing the obstacle is determined according to the floor adsorption level and the deposition state.

[0211] Optionally, when the influence range determining module 303 determines the influence range of each uncleanable obstacle according to the obstacle type and the scene image, it is used to:

[0212] Obtaining the cleaning range of the cleaning robot per unit time;

[0213] Determining an extension area according to the obstacle type and the on-site image;

[0214] An impact range is determined according to the cleaning range and the extended area.

[0215] Optionally, when the impact range determination module 303 determines the extension area according to the obstacle type and the scene image, it is used to:

[0216] Determining whether the obstacle is in motion according to the scene image;

[0217] If the object is in motion, the position of the obstacle in the scene image is analyzed according to the shooting time of the scene image;

[0218] Determine the movement direction and angle of the obstacle according to the position;

[0219] Determining a final position according to the movement direction and the movement angle;

[0220] According to the final position, an extension area is determined.

[0221] Optionally, when determining the extension area according to the final position, the impact range determination module 303 is used to:

[0222] Analyze the scene image to determine whether there is any garbage left in the obstacle;

[0223] If it exists, the type and scope of garbage retention are determined based on the image analysis results;

[0224] An extension area is determined according to the garbage retention type, the garbage retention range and the final location.

[0225] Optionally, when the influence range determining module 303 determines the influence range of each uncleanable obstacle according to the obstacle type and the scene image, it is used to:

[0226] Determining whether the uncleanable obstacle is movable according to the obstacle type;

[0227] If the mobility exists, determining the obstacle position of the uncleanable obstacle according to the on-site image;

[0228] According to the obstacle position, planning a convergence path;

[0229] According to the gathering path, the uncleanable obstacles are gathered and a gathering image is obtained;

[0230] The grouping image is analyzed to determine a grouping range, and the grouping range is determined as an influence range.

[0231] Optionally, when planning the convergence path according to the obstacle position, the impact range determination module 303 is used to:

[0232] Determining the infrastructure of the uncleanable obstacle based on the obstacle type;

[0233] Determining the cleaning characteristics of the uncleanable obstacle based on the infrastructure;

[0234] Determine the priority cleaning nodes according to the cleaning characteristics;

[0235] According to the priority cleaning nodes, a gathering path is planned.

[0236] Optionally, when the obstacle analysis module 302 determines whether there is an uncleanable obstacle according to the obstacle type, it is used to:

[0237] Determining the height of the obstacle according to the scene image;

[0238] Obtaining the cleaning properties of the cleaning robot;

[0239] Determining an obstacle crossing level of the cleaning robot according to the cleaning attributes;

[0240] According to the obstacle crossing level and the obstacle height, it is determined whether the obstacle is an uncleanable obstacle.

[0241] The system of this embodiment can be used to execute the method of any of the above embodiments. The implementation principles and technical effects are similar and will not be described in detail here.

Claims

1. A method for avoiding obstacles in a sweeping robot, characterized in that: The robot sweeping machine motion obstacle avoidance method is applied to the robot sweeping machine, the robot sweeping machine includes a radar scanning device, a camera device, and a control chip, the method is applied to the control chip, and the method includes: Obtaining the scanning results of the radar scanning device and the on-site images taken by the camera device; analyzing the scanning results and the on-site images to determine the types of obstacles in the area to be cleaned; Determine whether there is an uncleanable obstacle based on the obstacle type; If there is an uncleanable obstacle, judging whether there is interference between other obstacles and the uncleanable obstacle according to the scene image; if there is interference, determining the influence range of each uncleanable obstacle according to the obstacle type and the scene image; Determine interference conditions between interfering obstacles according to the impact range; According to the interference situation, the overlapping area of ​​each impact range is judged for cleaning feasibility, and avoidance is performed according to the cleaning feasibility and the impact range; The determining, according to the obstacle type and the scene image, the influence range of each uncleanable obstacle includes: Obtaining the cleaning range of the cleaning robot per unit time; Determining an extension area according to the obstacle type and the on-site image; Determine the impact range according to the cleaning range and the extended area; The step of determining the extension area according to the obstacle type and the scene image includes: Determining whether the obstacle is in motion according to the scene image; If the object is in motion, the position of the obstacle in the scene image is analyzed according to the shooting time of the scene image; Determine the movement direction and angle of the obstacle according to the position; Determining a final position according to the movement direction and the movement angle; Determining an extension area according to the final position; The step of determining the extension area according to the final position includes: Analyze the scene image to determine whether there is any garbage left in the obstacle; If it exists, the type and scope of garbage retention are determined based on the image analysis results; An extension area is determined according to the garbage retention type, the garbage retention range and the final location.

2. The method according to claim 1, characterized in that The analyzing the scanning result and the on-site image to determine the type of obstacles in the area to be cleaned includes: Analyzing the scanning results to determine the basic obstacle type; Analyzing the scene image to determine the state of obstacles; Determining the difficulty of clearing the obstacle according to the obstacle state; The obstacle type is determined according to the obstacle clearing difficulty and the basic obstacle type.

3. The method according to claim 2, characterized in that The basic obstacle type includes a liquid type, and determining the obstacle clearing difficulty according to the obstacle state includes: Analyzing the scene image to determine obstacle features; Determining the color of the liquid according to the obstacle characteristics; Retrieving a preset liquid stain database, and determining the liquid type according to the liquid color and the preset liquid stain database; determining whether liquid deposition exists according to the color of the liquid; If it is determined that liquid deposition is present, determining the floor adsorption level based on the type of liquid; The deposition state is determined according to the color of the liquid, and the difficulty of clearing the obstacle is determined according to the floor adsorption level and the deposition state.

4. The method according to claim 1, characterized in that: The determining, according to the obstacle type and the scene image, the influence range of each uncleanable obstacle includes: Determining whether the uncleanable obstacle is movable according to the obstacle type; If the mobility exists, determining the obstacle position of the uncleanable obstacle according to the on-site image; According to the obstacle position, planning a convergence path; According to the gathering path, the uncleanable obstacles are gathered and a gathering image is obtained; The grouping image is analyzed to determine a grouping range, and the grouping range is determined as an influence range.

5. The method according to claim 4, characterized in that The planning of the convergence path according to the obstacle position includes: Determining the infrastructure of the uncleanable obstacle based on the obstacle type; Determining the cleaning characteristics of the uncleanable obstacle based on the infrastructure; Determine the priority cleaning nodes according to the cleaning characteristics; According to the priority cleaning nodes, a gathering path is planned.

6. The method according to claim 1, characterized in that The determining, according to the obstacle type, whether there is an uncleanable obstacle includes: Determining the height of the obstacle according to the scene image; Obtaining the cleaning properties of the cleaning robot; Determining an obstacle crossing level of the cleaning robot according to the cleaning attributes; According to the obstacle crossing level and the obstacle height, it is determined whether the obstacle is an uncleanable obstacle.

7. A sweeping robot motion obstacle avoidance system, characterized in that: The method as claimed in any one of claims 1 to 6 comprises: A type analysis module is used to obtain the scanning results of the radar scanning device and the on-site images taken by the camera device; analyze the scanning results and the on-site images to determine the type of obstacles in the area to be cleaned; An obstacle analysis module is used to determine whether there is an uncleanable obstacle according to the obstacle type; An influence range determination module, for determining, if an uncleanable obstacle exists, whether interference exists between other obstacles and the uncleanable obstacle according to the scene image, and if interference exists, determining the influence range of each uncleanable obstacle according to the obstacle type and the scene image; An interference situation determination module, used to determine the interference situation between the interfering obstacles according to the influence range; The avoidance range determination module is used to determine the feasibility of cleaning the overlapping areas of each impact range according to the interference situation, and to avoid according to the cleaning feasibility and the impact range.

Images