Control method and control device of mobile cleaning device, device and storage medium

Abstract

The application provides a control method and device of a mobile cleaning device, equipment and a storage medium. The method comprises: obtaining sensing information of a path during movement of the mobile cleaning device in a working area; confirming an obstacle distribution during movement based on the sensing information, and obtaining an obstacle map of the working area according to the obstacle distribution during movement; and in response to a call instruction of the mobile cleaning device, calling the obstacle map to control the mobile cleaning device to avoid obstacles during movement. The above method can solve the problems of obstacle avoidance failure and work interruption when the mobile cleaning device fails to sense.

Description

Technical Field

[0001] This application relates to the field of mobile cleaning equipment, and more particularly to a control method, control device, equipment, and storage medium for mobile cleaning equipment. Background Technology

[0002] Mobile cleaning equipment is capable of autonomously moving and completing cleaning tasks in various environments, requiring obstacle detection and avoidance. It primarily relies on a real-time visual perception + real-time path planning operating mode. For example, visual sensors identify obstacles within the work area in real time, and the path is dynamically adjusted based on the detected obstacles to achieve obstacle avoidance. However, in some scenarios, the mobile cleaning equipment's perception may fail, resulting in an inability to reliably detect obstacles, leading to obstacle avoidance failures and operation interruptions. Summary of the Invention

[0003] The main purpose of this application is to provide a control method, control device, equipment and storage medium for mobile cleaning equipment, which aims to solve problems such as obstacle avoidance failure and operation interruption when the mobile cleaning equipment fails to sense the problem.

[0004] In a first aspect, this application provides a control method for a mobile cleaning device, comprising: As the mobile cleaning device moves through the work area, it acquires path perception information; Based on the perceived information, the distribution of obstacles during the movement is confirmed, and an obstacle map of the work area is obtained based on the distribution of obstacles during the movement. In response to the retrieval command of the mobile cleaning device, the obstacle map is retrieved to control the mobile cleaning device to avoid obstacles during its movement.

[0005] Secondly, this application also provides a control device, which includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the control method as described in any one of the embodiments of this application. Thirdly, this application also provides a mobile cleaning device, which includes a body, a sensing sensor, and a control device as described in the embodiments of this application. The sensing sensor is used to acquire path perception information; the sensing sensor and the control device are disposed on the body.

[0006] Fourthly, this application also provides a storage medium for computer-readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the control method as described in any one of the embodiments of this application.

[0007] This application provides a control method for a mobile cleaning device. During the movement of the mobile cleaning device within a work area, the method acquires path perception information; based on the perception information, it confirms the distribution of obstacles during movement and obtains an obstacle map of the work area based on this distribution; in response to a retrieval command from the mobile cleaning device, it retrieves the obstacle map to control the device to avoid obstacles during movement. Therefore, even in the event of perception failure, the mobile cleaning device can still avoid obstacles using the obstacle distribution in the obstacle map. This solves problems such as obstacle avoidance failure and work interruption that occur when the mobile cleaning device's perception fails, thereby improving the efficiency and reliability of the mobile cleaning device. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic block diagram of a mobile cleaning device provided in an embodiment of this application; Figure 2 This is another schematic block diagram of the mobile cleaning device provided in the embodiments of this application; Figure 3 This application provides a schematic flowchart of the steps of a control method for a mobile cleaning device. Figure 4 This is a schematic diagram of a scenario with multiple sub-work areas provided in an embodiment of this application; Figure 5 for Figure 3 A flowchart illustrating a sub-step of the control method in the process; Figure 6 for Figure 3 A flowchart illustrating another sub-step of the control method in the process; Figure 7 A flowchart illustrating the steps of another control method for a mobile cleaning device provided in this application embodiment; Figure 8 A flowchart illustrating the steps of a control method for a lawnmower as a mobile cleaning device provided in an embodiment of this application; Figure 9This is a schematic block diagram of a control device provided in an embodiment of this application.

[0010] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0013] In the autonomous navigation and operation of mobile cleaning equipment (such as lawnmowers, robotic vacuum cleaners, and cleaning vehicles), real-time environmental perception and obstacle avoidance are among the core technologies for achieving efficient and safe operation. Traditional solutions mainly rely on visual sensors (such as cameras and LiDAR) to collect environmental information in real time, detect obstacles through image recognition or point cloud processing technology, and adjust the travel path accordingly. However, visual perception is easily affected by environmental factors. For example, at night or in low-light conditions, the accuracy of visual perception drops significantly, which may lead to distortion or interruption of perception information. Once visual perception fails, the mobile cleaning equipment may not be able to identify obstacles in time, causing collision risks, operation interruptions, or repeated cleaning, reducing work efficiency and reliability.

[0014] Based on this, the embodiments of this application provide a control method, control device, mobile cleaning device and storage medium for a mobile cleaning device. The embodiments of this application can provide a redundant obstacle avoidance mechanism, which can still ensure the continuous obstacle avoidance capability of the mobile cleaning device when visual perception is abnormal, and is applicable to various mobile cleaning scenarios such as home, commercial and industrial.

[0015] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0016] Figure 1 This is a schematic block diagram of a mobile cleaning device provided in an embodiment of this application. Figure 1 As shown, the control method of the mobile cleaning device can be applied to the mobile cleaning device 10, which includes a body 11 and a sensing sensor 12.

[0017] The main body 11 can be the physical entity and functional platform of the mobile cleaning device 10, for example, the sensing sensor 12 can be installed on the main body 11. The main body 11 can also be equipped with accessories such as a drive system, a cleaning system, and an energy system. For example, the drive system can include a motor, wheels (or tracks), and a transmission mechanism, responsible for driving the mobile cleaning device 10 to move, turn, and stop within the working area according to control commands. The cleaning system can integrate brushes, a vacuuming device, a water tank, a rag, etc., responsible for performing the actual cleaning tasks. The energy system can include a rechargeable battery to provide power to the mobile cleaning device 10 (including the drive system, cleaning system, etc.).

[0018] The sensing sensor 12 can be used to detect sensing information in the direction of movement of the mobile cleaning device 10. For example, the sensing sensor 12 can use active or passive detection technology to detect space in the direction of movement, thereby obtaining sensing information about the path. The sensing sensor 12 can be a single-modal or multi-modal sensor, and can include at least one of the following: a visual sensor, an ultrasonic sensor, an infrared sensor, or a lidar. The mobile cleaning device 10 can be an intelligent device responsible for outdoor cleaning work, such as a lawnmower, snowplow, sweeper, or other mobile cleaning robot; the mobile cleaning device 10 can also work indoors.

[0019] In one embodiment, such as Figure 2 As shown, the mobile cleaning device 10 also includes a control device 13, with both the sensing sensor 12 and the control device 13 located on the main body 11. The control method can be as follows: during the movement of the mobile cleaning device 10 within the work area, the sensing sensor 12 acquires path perception information. The control device 13 confirms the distribution of obstacles during the movement based on the path perception information and obtains an obstacle map of the work area based on this distribution. During subsequent cleaning, the control device 13 retrieves the obstacle map in response to a retrieval command from the mobile cleaning device 10. Then, based on the obstacle map or in conjunction with some of the sensing sensors 12 that are still functioning normally, the mobile cleaning device 10 can be controlled to avoid obstacles during its movement.

[0020] Those skilled in the art will understand that Figure 1 and Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the embodiments of this application, and does not constitute a limitation on the mobile cleaning device applied thereto in the embodiments of this application. A specific mobile cleaning device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0021] The following will combine Figure 1 and Figure 2The illustrated structure provides a detailed description of the control method for the mobile cleaning device provided in the embodiments of this application. It should be noted that... Figure 1 and Figure 2 The structure shown is only used to explain the control method of the mobile cleaning device provided in the embodiments of this application, but does not constitute a limitation on the application scenarios of the control method of the mobile cleaning device provided in the embodiments of this application.

[0022] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating the steps of a control method for a mobile cleaning device provided in an embodiment of this application. This control method can be applied to a control device, which can be installed on the mobile cleaning device.

[0023] like Figure 3 As shown, the control method for the mobile cleaning device includes steps S101 to S103.

[0024] Step S101: Acquire path perception information as the mobile cleaning device moves through the work area.

[0025] The perceived information refers to the sensor data generated by the mobile cleaning equipment through physical detection of the external environment. This perceived information can be acquired through sensors mounted on the mobile cleaning equipment. These sensors can be single-modal or multi-modal, and are used at least to detect the direction of movement of the mobile cleaning equipment. The perceived information can include obstacle information, as well as terrain features (such as slope and texture) or dynamic object trajectories. This perceived information provides the initial input for subsequent obstacle distribution. Different types of sensors have their own advantages and limitations; multi-modal sensors can complement each other, improving the accuracy and reliability of path perception.

[0026] For example, a mobile cleaning device is a lawnmower. The lawnmower's working area can be the lawn area to be mowed, and its boundaries can be garden paths, flower beds, trees, or artificially defined boundary lines (such as the boundary lines of a wired lawnmower). Obstacles in the lawnmower's working area can include, but are not limited to, trees, shrubs, flower beds, toys, pets, pedestrians, and patio furniture. The lawnmower's perception principle can be based on multi-sensor data fusion. For example, the control device can simultaneously process data from perception sensors such as cameras and LiDAR. The image data collected by the camera can be semantically segmented using computer vision algorithms to identify pixel areas such as "grass," "trees," and "roads." The point cloud data collected by LiDAR can provide accurate distance information and generate the outlines of obstacles. The fusion algorithm can combine the data collected by the camera and LiDAR to form a perception information that includes both geometric and semantic information.

[0027] Step S102: Based on the perception information, confirm the distribution of obstacles during the movement, and obtain an obstacle map of the work area based on the distribution of obstacles during the movement.

[0028] The obstacle distribution can be a spatiotemporal distribution with probabilities or confidence levels. For example, the obstacle distribution could record "there is a 95% probability that a static obstacle (tree) exists at coordinates (X1, Y1)," while "a moving object is detected at coordinates (X2, Y2), with a trajectory of...". The obstacle distribution can be further classified into permanent obstacles (such as walls) and temporary obstacles (such as moving chairs), and its spatiotemporal attributes can be recorded to support dynamic map updates.

[0029] Among them, the obstacle map can be a quantitative environmental model formed by spatially registering and structurally modeling the obstacle distribution information using the positioning data of mobile cleaning equipment as a spatial reference.

[0030] It's important to note that perceived information can be processed using algorithms (such as clustering analysis, semantic segmentation, or Bayesian filtering) to identify the location, size, and type of obstacles (such as static shrubs or dynamic pets). For example, clustering algorithms can be used to group sensor data points and distinguish obstacle boundaries, or semantic segmentation models (such as convolutional neural networks) can be used to differentiate between passable and obstacle areas. The identified obstacle distribution can then be fused with the user's own location data (such as GPS or UWB positioning) to construct an obstacle map represented as a grid map or topological map. For instance, a grid map divides the working area into cells, each labeled "obstacle" or "free," and stores the probability or confidence level of the obstacle.

[0031] For example, confirming the distribution of obstacles during movement based on perception information can include the following steps: First, align and fuse perception information acquired by different sensors at different times to form a consistent environmental description. For example, match the semantic label of "flower bed" identified by the camera with the precise outline and location coordinates of the flower bed measured by LiDAR. Then, using machine learning models (such as convolutional neural networks CNN) or traditional computer vision algorithms, identify obstacles from the fused data and classify them (e.g., static obstacles - trees / flower beds; dynamic obstacles - pets; temporary obstacles - garden chairs). Afterward, by combining the mobile cleaning device's own positioning information (obtained through GPS, odometer, or SLAM algorithms), determine the precise location (X, Y coordinates) and size of each identified obstacle in the global coordinate system, thereby obtaining the obstacle distribution of the mobile cleaning device during movement.

[0032] In one embodiment, the generation of the obstacle map may rely on multi-source data fusion (such as sensor data and positioning data) and reduce uncertainty through probabilistic models (such as Kalman filtering). For example, the location of obstacles in the working area may be improved in accuracy through Bayesian updates based on multiple observations, avoiding single-perception errors.

[0033] In one embodiment, the work area is configured as multiple sub-work areas; obtaining an obstacle map of the work area based on the distribution of obstacles during the movement includes: generating a temporary obstacle map of each sub-work area based on the distribution of obstacles during the movement of each sub-work area; and obtaining an obstacle map of the work area based on the temporary obstacle map of each sub-work area.

[0034] Multiple sub-work areas can be derived from dividing the entire work area (such as a large courtyard). For example, the work area can be divided into multiple grids, with each grid representing a sub-work area. Each sub-work area can have a unique identifier (ID) and boundary definition. The sizes of the multiple sub-work areas can be the same or different; for example, a work area exceeding a certain area can be automatically divided into several sub-work areas of similar size. The division of multiple sub-work areas can be completed automatically during the initial exploration by the mobile cleaning device, or it can be manually marked by the user through the app.

[0035] It's important to note that building temporary obstacle maps for individual sub-work areas requires significantly less computation and memory compared to constructing the entire obstacle map at once. This allows the algorithm to run on mobile clearing devices with limited computing resources. For example, when a lawnmower is working within a specific sub-work area, it can focus on building a temporary obstacle map for that area. Within a single sub-work area, due to the limited travel distance, the cumulative error of the obstacle map is kept within a small range, resulting in high internal accuracy for each sub-work area's temporary obstacle map. During global stitching, even if slight pose errors exist between temporary obstacle maps, the optimization algorithm can distribute these errors evenly across the entire obstacle map, rather than allowing errors to accumulate along a single long path. This effectively avoids severe map distortion.

[0036] For example, when a lawnmower builds a map of a 1000-square-meter yard, if it divides it into 10 sub-working areas of 100 square meters each, the positioning error within each sub-working area may only be a few centimeters. During stitching, an optimization algorithm is used to correct the error, resulting in a final global obstacle map with higher accuracy than one built by traversing the entire 1000-square-meter area in one go.

[0037] In one embodiment, the mobile cleaning device includes a lawnmower. The area of ​​each sub-working area is less than or equal to the full-box mowing area of ​​the lawnmower, where the full-box mowing area characterizes the mowing area corresponding to the collection capacity of the lawnmower's collection box. The area of ​​the sub-working area can be a preset value (e.g., "under standard grass conditions, the collection box can hold the grass clippings generated from mowing 50 square meters," thus determining the preset value of the sub-working area to be less than or equal to 50 square meters), or it can be dynamically adjusted based on historical data and the current grass density. By setting the area of ​​each sub-working area to be less than or equal to the maximum working area that the lawnmower can complete each time it empties its collection box, seamless and efficient execution of mowing tasks is achieved, maximizing the effective working efficiency within a single charging cycle.

[0038] It's important to note that before the task begins, if the map of the entire work area is known (whether through initial exploration or user import), the lawnmower can automatically mesh the entire work area into several sub-work areas of equal size, using the full mowing box area as a reference dimension. If a work area (such as the backyard) is too large, it can be divided into multiple suitable sub-work areas. When exploring and mapping in an unknown environment, when the lawnmower determines that the area of ​​the explored continuous region is close to the full mowing box area, it can end the exploration and map building of the current sub-work area, mark it as a complete sub-work area, and then return to the base station to empty the collection box.

[0039] For example, the lawnmower's collection box can mow 50 square meters when fully loaded. Figure 4 As shown, the yard is divided into four 50-square-meter sub-work areas (A, B, C, D). After mowing sub-work area A, the lawnmower 21 can return to base station 22 to empty the collection box even if it is not full, and then proceed to sub-work area B to continue working. This completely avoids the scenario of stopping midway through mowing sub-work area B due to a full collection box. Furthermore, passive interruptions can occur in any corner of the work area, and the path back to base station 22 for cleaning can be long and inefficient. Therefore, when planning the work paths for each sub-work area (A, B, C, D), the endpoint can be set near base station 22 or the main road. This minimizes the time and energy consumed by the mowing task. For example, when planning the mowing path for sub-work area C, the endpoint can be arranged as close as possible to the main road of the yard. After completing sub-work area C, the lawnmower 21 can take the shortest path to the main road and return to base station 22 efficiently.

[0040] In one embodiment, obtaining an obstacle map of a work area based on a temporary obstacle map of each sub-work area includes: after the work in each sub-work area is completed, obtaining a temporary obstacle map of each sub-work area to determine a map for each sub-work area.

[0041] It should be noted that once the work in a sub-work area is completed, the exploration of that area can be considered sufficient, and the temporary obstacle map at this point is sufficiently complete. Therefore, the current temporary obstacle map can be locked or marked as the final version, i.e., designated as the sub-work area map. Each confirmed sub-work area map can be stored in the mobile cleaning device's memory (such as non-volatile memory). The next time the mobile cleaning device operates, these confirmed obstacle maps can be directly loaded without rebuilding, thereby improving work efficiency.

[0042] For example, the mobile cleaning device has already determined the maps for each sub-work area during the day. When the user restarts the mobile cleaning device at night or the next day, it can directly load the confirmed maps for each sub-work area, resulting in extremely high overall efficiency. In this case, if a potted plant is found in the corner of the side yard (sub-work area C), the obstacle map for sub-work area C can be updated locally without rebuilding the obstacle map for the entire yard (work area). Therefore, it will not affect the stability of the confirmed obstacle maps for the front yard, back yard, and other sub-work areas. Similarly, if the mobile cleaning device is interrupted due to insufficient power while building a map for a certain sub-work area, it only needs to replenish the obstacle map for that work area next time and then reassemble it, without affecting the other sub-work area maps that have already been built.

[0043] In one embodiment, obtaining an obstacle map of the work area based on the temporary obstacle map of each sub-work area includes merging the temporary obstacle maps of each sub-work area to obtain an obstacle map of the work area.

[0044] It's important to note that the core objective of merging temporary obstacle maps from each sub-work area is to identify the spatial transformation relationships between these maps and unify them into a single global coordinate system. The merging process can involve two stages: map registration and map fusion. Map registration aims to find the relative data (i.e., rotation and translation parameters) between two adjacent or overlapping temporary obstacle maps. Map fusion, following registration, integrates the data from multiple temporary obstacle maps into a single global obstacle map. For raster maps, this can involve fusing raster values ​​(such as occupancy probabilities) from overlapping areas. Since each temporary obstacle map is constructed within a finite area, its internal cumulative error is small, resulting in high accuracy. Therefore, merging temporary obstacle maps from each sub-work area can significantly improve obstacle map accuracy in large-scale environments.

[0045] Step S103: In response to the retrieval command of the mobile cleaning device, retrieve the obstacle map to control the mobile cleaning device to avoid obstacles during its movement.

[0046] The mobile cleaning device can retrieve obstacle maps using commands that can be triggered automatically or manually by the user. Automatic triggering can include detecting at least one of the following: no camera signal, abnormal LiDAR data, or the confidence level of key sensor data falling below a safety threshold. It can also include situations such as suddenly entering a dark environment (visual impairment) or dense fog / heavy rain (impact on LiDAR). Manual triggering can include the user forcing the mobile cleaning device to retrieve obstacle maps for navigation via an application.

[0047] It should be noted that the control device can control the mobile cleaning equipment to avoid obstacles during its movement based on the retrieved obstacle map. Specifically, it can perform global path replanning, such as immediately interrupting the original path that may be based on a failed real-time perception system and replanning a safe path from the current position to the target point based on the obstacle map. The planning algorithm (such as A* and D* algorithms) can avoid all known obstacles marked on the obstacle map. It can also perform local behavior control, such as generating specific control commands (such as differential speed between the left and right wheels) to drive the main body of the mobile cleaning equipment (such as the motor and wheels of the main body 11) to perform actions such as steering, deceleration, or stopping to follow the new safe path. For example, real-time perception obstacle avoidance can serve as the primary perception system, and the obstacle map can serve as the backup perception system. When the primary perception system fails, the backup perception system seamlessly takes over, ensuring that the core function (obstacle avoidance) is not lost. In abnormal situations where the main sensing system fails completely or partially, the obstacle avoidance capability of the mobile cleaning equipment can still be maintained by the obstacle distribution in the obstacle map. Therefore, it can solve problems such as obstacle avoidance failure and operation interruption when the mobile cleaning equipment's sensing fails, ensuring the continuity of the operation and the safety of the mobile cleaning equipment, thereby improving the working efficiency and reliability of the mobile cleaning equipment.

[0048] For example, when a lawnmower is mowing a yard, the camera gets obscured by splashes of mud. The control unit detects the visual failure and immediately retrieves the obstacle map of the yard from its memory. At this moment, there is a large tree 10 meters in front of the lawnmower, so the control unit starts to move around it in advance, and the operation does not need to be interrupted.

[0049] In other words, when real-time sensing fails, the mobile mower does not go blind; instead, it switches to relying on spatial memory (obstacle map) to recall the distribution of obstacles in the environment. At this point, the obstacle map serves as a reliable reference, allowing obstacle avoidance decisions to continue. Therefore, it can solve problems such as obstacle avoidance failure and operation interruption that occur when the mobile mower's sensing fails. Because the obstacle map is pre-stored and quickly retrieved, the entire switching process can be completed within milliseconds, allowing the lawnmower to continue moving without stopping.

[0050] In one embodiment, the retrieval command is triggered based on abnormal environmental conditions in which the mobile cleaning device is located. Abnormal environmental conditions include at least one of the following: low-light environment, high-light environment, and severe weather environment. In this case, the retrieval command is a passive response driven by environmental changes. For example, environmental parameters are continuously monitored by sensors, and when the parameters exceed a preset normal range, it is determined to be an abnormal environmental condition, and a retrieval command is automatically generated.

[0051] It's important to note that low-light environments primarily affect visual sensors (such as cameras). For example, at dusk, at night, or when mobile cleaning equipment enters shaded or indoor areas, the signal-to-noise ratio of the images captured by the camera drops sharply, becoming blurry and dim, causing vision-based object recognition and feature tracking algorithms to fail. Therefore, insufficient lighting can be detected by using a light sensor or by analyzing the image brightness histogram, indicating a decrease in the reliability of visual perception, thereby triggering a command to switch to a navigation mode that relies on an obstacle map.

[0052] It should be noted that strong light environments also affect visual sensors. For example, direct sunlight at midday, reflections from water surfaces or glass curtain walls can cause image overexposure or glare, obscuring key feature information. Therefore, when a large area of ​​saturated pixels or abnormally high brightness values ​​are detected in an image, it is determined to be strong light interference, making visual perception unreliable and triggering a retrieval command.

[0053] It should be noted that severe weather conditions, such as rain, snow, and fog, have a significant impact on lidar. Water droplets, ice crystals, or dust in the air absorb and scatter the laser beam, causing the point cloud data to become sparse and noisy, resulting in a significant decrease in ranging accuracy and effective range. They can also obstruct camera lenses. Furthermore, strong winds can affect the stability of mobile cleaning equipment, indirectly affecting the accuracy of odometers and challenging the positioning accuracy of SLAM. Therefore, severe weather can be detected by accessing meteorological information, using humidity sensors, or directly analyzing the quality of the lidar point cloud (e.g., abnormally low point cloud density). When the performance of the main sensing sensors (especially lidar) severely degrades, a retrieval command is triggered.

[0054] For example, a smart lawnmower can work at night. When it detects low light and the camera can no longer clearly identify the boundaries, it automatically retrieves a high-precision map of yard obstacles and uses LiDAR or ultrasound for assisted positioning to continue mowing, achieving all-weather operation.

[0055] For example, a lawnmower can operate in light rain. Due to raindrop interference, the effective detection range of the lidar is reduced from 20 meters to 5 meters. At this point, a command can be triggered, and the mower moves to avoid known flower beds and lampposts based on an obstacle map. For newly appearing obstacles within 5 meters (such as a bucket), emergency obstacle avoidance is achieved using degraded but still usable lidar and ultrasonic sensors.

[0056] Understandably, the sensing range and quality of lidar and cameras will decrease in inclement weather, but most static obstacles in the environment (such as trees and building boundaries) remain unchanged. Accessing the obstacle map essentially provides the mobile cleaning device with a memorized view, enabling it to avoid these permanent obstacles and ensuring basic safety. At this point, the mobile cleaning device can focus on dealing with potential dynamic obstacles not recorded in the obstacle map (such as people and pets) and can adopt more conservative strategies (such as reducing speed).

[0057] In one embodiment, the retrieval command is triggered based on an abnormal state of the sensing sensor used to acquire sensing information. The abnormal state includes at least one of the following: malfunction, obstruction by a foreign object, or unavailable sensing information. In this case, the retrieval command may be automatically triggered when the sensor data is abnormal or the confidence level is below a threshold.

[0058] It should be noted that a malfunction can be a complete failure of the sensor hardware. Examples include a LiDAR motor stopping, damage to the internal circuitry of a camera module, or a transducer failure in an ultrasonic sensor. This can be determined by checking the sensor's heartbeat signal, status register, or whether the communication link is interrupted. For instance, if no data packets are received from a sensor within a predetermined time, that sensor is considered to have malfunctioned.

[0059] It's important to note that obstruction by foreign objects can refer to the physical blockage of the sensor's detection window. For example, the camera lens or lidar window of a lawnmower / sweeper might be covered in dirt, grass clippings, or dust. For cameras, this can be determined by analyzing the image for large, fixed-pattern color blocks or blurred areas (e.g., the entire image turning green or becoming completely blurry). For lidar / ultrasonic sensors, if the sensor consistently returns a fixed distance value at close range, or returns an "invalid value," it indicates that the window is completely obstructed.

[0060] It should be noted that unusable sensing information refers to data output by sensors that cannot be used for effective environmental perception due to some reason. Unusable sensing information can include severely distorted data, communication errors, and algorithm failure. For example, severely distorted data might be a LiDAR system returning a chaotic point cloud when viewed against a highly reflective surface (such as a mirror). Communication errors could be caused by a large number of bit errors during data transmission, making data packets unreadable. Algorithm failure could be caused by the existence of raw data, but subsequent processing algorithms (such as feature extraction and object recognition) fail continuously due to poor data quality, resulting in the inability to output effective sensing information.

[0061] For example, one of the ultrasonic sensors of a robotic vacuum cleaner was completely blocked by a sock. The control device detected that the ultrasonic sensor was continuously returning a fixed value of 10cm (abnormal), and determined that it was blocked and the data was invalid. Subsequently, the control device retrieved an obstacle map and relied more on other normal sensors (such as an ultrasonic sensor at another angle or a collision sensor) for assistance, successfully avoiding collisions with furniture due to relying on incorrect data.

[0062] In one embodiment, after the retrieval command is triggered, an alarm message can also be sent to the user to indicate the presence of danger, thereby avoiding situations such as the mobile cleaning device getting stuck, which could lead to a more complicated rescue situation.

[0063] For example, when a lawnmower is working, the camera lens gradually becomes covered with mud. Upon detecting a decrease in image clarity, the system retrieves a map of obstacles in the work area and notifies the user that "the vision system needs cleaning." After the user cleans the area promptly, moving the cleaning device restores full functionality, resulting in a short overall task interruption time.

[0064] In one embodiment, such as Figure 5 As shown, after retrieving the obstacle map (i.e., step S104), steps S1041 to S1042 are also included: Step S1041: Based on the path perception information obtained again by the mobile cleaning device, confirm the distribution of new obstacles during the journey.

[0065] Step S1042: Update the obstacle map according to the new obstacle distribution.

[0066] It's important to note that the new obstacle distribution can include newly added obstacles, such as a new chair or toy placed in the yard. It can also include obstacles that have disappeared, such as a temporary box marked on the obstacle map being moved. For newly added obstacles, they can be added to the corresponding location on the global obstacle map and assigned an initial confidence level. For disappeared obstacles, their confidence level can be lowered, or they can be removed. For example, if an obstacle is not detected for several consecutive time windows, it can be removed from the obstacle map. Updating the obstacle map based on the new obstacle distribution allows the mobile cleaning device to verify and update the obstacle map each time it operates, ensuring that its obstacle avoidance decisions are based on the latest environmental information.

[0067] For example, a lawnmower navigates at night using an obstacle map. When its ultrasonic sensors detect a new obstacle (such as a watering can) on a previously open path in the obstacle map, a temporary marker can be made at that location, and the lawnmower can be guided to avoid it. The next day, even if the vision sensors recover, the presence of this new obstacle will be recognized.

[0068] In one embodiment, the obstacle map includes multiple sub-work area maps; updating the obstacle map according to the new obstacle distribution includes: determining the sub-work area map corresponding to the new obstacle distribution; and updating the corresponding sub-work area map according to the new obstacle distribution.

[0069] It's important to note that since the global obstacle map can be composed of multiple sub-work area maps, it's necessary to maintain the boundary information (positional range in the global coordinate system) of each sub-work area map. Once a new obstacle distribution is confirmed (e.g., a new obstacle is found with global coordinates (X, Y)), a simple spatial query can determine which sub-work area map this coordinate point falls within, thus identifying the sub-work area map that needs updating. The specific update operation can be marking the grid corresponding to the newly added obstacle as "occupied" or marking the grid of a disappeared obstacle as "free." Simultaneously, the confidence level of these grids can be updated. Modularizing the global obstacle map into multiple sub-work area maps ensures that maintaining and updating one sub-work area will not affect other modules, reducing maintenance complexity and coupling. Furthermore, it enables high efficiency and low overhead in obstacle map updates while guaranteeing the consistency of the global obstacle map data.

[0070] In one embodiment, such as Figure 6 As shown, after retrieving the obstacle map (i.e., step S104), steps S1051 to S1052 are also included: Step S1051: Based on the obstacle map, plan the recharge path for the mobile cleaning equipment.

[0071] Step S1052: During the process of the mobile cleaning device traveling along the recharge path, in response to the confirmation that there are obstacles on the recharge path, the obstacle map is updated and the recharge path is replanned.

[0072] It's important to note that when the mobile cleaning device needs to return to its charging dock (e.g., due to low battery triggering or task completion), the control unit first retrieves the latest global obstacle map. Then, using a path planning algorithm (such as the A algorithm or DLite algorithm), it calculates the optimal path from the mobile cleaning device's current location to the charging dock. This path avoids all known static obstacles in the obstacle map. While the mobile cleaning device travels along the planned path, some of its sensing sensors remain active. If it detects an obstacle not recorded in the obstacle map (such as a temporarily placed box or a playing child), a new obstacle distribution is confirmed. The information about this small obstacle is updated in the global obstacle map (specifically to the corresponding sub-working area). Then, based on the updated obstacle map, starting from the current location, a new return-to-charging path is planned that avoids all known obstacles (including this newly discovered one).

[0073] Therefore, updating the obstacle map and replanning the recharge path improves the success rate and reliability of automatic recharge tasks, ensuring that the mobile cleaning device can return to its charging station efficiently and safely under any circumstances. In some cases, the end of the recharge path (near the charging dock) is a high-risk area for dynamic obstacles (such as a chair being pulled out or a pet lying on the road). If the mobile cleaning device stubbornly follows its original plan, it will inevitably fail. The dynamic replanning mechanism allows it to flexibly respond to such unexpected situations, ultimately resulting in successful docking.

[0074] For example, when a lawnmower is low on battery and returning to its charging dock, it might find a child's toy car blocking its path when it's 2 meters away. The obstacle map can be updated immediately, and a new path can be planned to bypass the toy car, allowing it to successfully zigzag around and dock for charging. Without this feature, the lawnmower could repeatedly try to bump into the toy car, eventually running out of power.

[0075] In one embodiment, such as Figure 7 As shown, after retrieving the obstacle map (i.e., step S104), step S106 is also included.

[0076] Step S106: In response to the detection of an error in the obstacle map, delete the obstacle map.

[0077] The deletion operation can involve removing the obstacle map files from the mobile cleaning device's non-volatile memory and clearing the cached obstacle map in memory. It's important to note that after deleting the obstacle map, the mobile cleaning device can enter a safe degraded mode. This could include switching to a conservative obstacle avoidance mode based solely on real-time perception (such as walking along edges and stopping upon encountering any obstacle), or sending an emergency notification to the user, reporting a map error and requesting intervention.

[0078] In one embodiment, detecting an error in the obstacle map includes: acquiring collision information with obstacles during the movement of the mobile cleaning device. The collision information can be used to characterize events that occur when the mobile cleaning device comes into physical contact with physical entities in the external environment.

[0079] It should be noted that obtaining collision information with an obstacle can refer to the mobile cleaning device directly detecting physical contact with the obstacle through collision sensors (such as buffer sensors and their underlying microswitches, touch sensors, or collisions inferred by detecting abnormal increases in motor torque). Collision information can include the location (front left, front right, etc.) and intensity of the collision.

[0080] For example, a lawnmower's obstacle map might not mark a transparent glass greenhouse barrier. LiDAR might penetrate the barrier, and cameras might miss it. However, if the collision sensor detects a solid obstacle not on the obstacle map when the lawnmower hits the barrier, it confirms an error in the obstacle map.

[0081] In one embodiment, detecting an error in the obstacle map includes: the mobile cleaning device acquiring an obstacle not recorded on the obstacle map during its movement. It should be noted that acquiring an obstacle not recorded on the obstacle map can mean that the mobile cleaning device's sensing sensors (such as LiDAR, cameras, or ultrasonic sensors) detect a real object while scanning the environment in real time, and the object's location is marked as free or unknown on the currently retrieved obstacle map.

[0082] In other words, the obstacle was indeed detected by the sensors, but it wasn't in the existing obstacle map. This indicates that the environment has changed, but the obstacle map hasn't been updated. When real-time perception data has high confidence, it should be trusted first, and the obstacle map should be deemed incorrect. This allows for proactive, preventative map error detection, identifying and correcting incompleteness in the obstacle map before a potential collision occurs, thus achieving a higher level of safety and intelligence.

[0083] For example, a lawnmower is traveling along a path planned by an obstacle map when its LiDAR scans a toy box 3 meters ahead that is not recorded on the obstacle map. The control unit can determine that there is an omission error in the obstacle map and update the location of the toy box to the obstacle map.

[0084] The control method for the mobile cleaning equipment provided in the above embodiments acquires path perception information during the movement of the mobile cleaning equipment in the work area; confirms the distribution of obstacles during the movement based on the perception information, and obtains an obstacle map of the work area based on the obstacle distribution; and retrieves the obstacle map in response to a retrieval command from the mobile cleaning equipment to control the mobile cleaning equipment to avoid obstacles during movement. Therefore, even in the case of visual perception failure, obstacle avoidance can be achieved using the obstacle distribution in the obstacle map, thus solving problems such as obstacle avoidance failure and work interruption that occur when visual perception fails, thereby improving the working efficiency and reliability of the mobile cleaning equipment.

[0085] This application's embodiments construct a backup obstacle avoidance channel in addition to vision and other perception sensors through a closed loop of "perception-mapping-retrieval". When real-time perception fails, traditional solutions may completely lose obstacle avoidance functionality due to the lack of backup data sources. However, this method provides prior knowledge of the environmental structure through a pre-stored obstacle map. Even if real-time perception is interrupted, the mobile cleaning device can still avoid risks based on the obstacle distribution in the obstacle map. Its core lies in decoupling real-time perception from obstacle map data: the perception sensor is responsible for dynamic environmental response, while the obstacle map serves as a prior knowledge base for the static environment. For example, if the vision system fails due to low light, the wall positions recorded in the obstacle map can be directly used for path replanning to avoid collisions. This redundancy mechanism significantly reduces the probability of operation interruption due to single-point failure.

[0086] The obstacle map can be iteratively updated multiple times during the operation of the mobile cleaning equipment (e.g., through incremental learning), gradually improving the environmental model. Once visual perception is restored, the differences between real-time data and the obstacle map can be compared, dynamically correcting the obstacle map's content. This adaptive learning capability enables the mobile cleaning equipment to cope with environmental changes (such as toy movement), avoiding misjudgments caused by outdated obstacle maps. In the long term, it exhibits stronger robustness in complex environments, reducing failures caused by environmental dynamics.

[0087] For example, when the mobile cleaning device is a lawnmower, the embodiments of this application can be used in the field of autonomous navigation and environmental perception control technology for intelligent lawnmowers, specifically applied to scenarios such as static obstacle avoidance, all-weather operation, and map management optimization for intelligent lawnmowers. The core logic can be: daytime area-based mapping → after a single area is completely mowed, the obstacle map is permanently stored → at night / for the next operation, the historical map is directly reused for obstacle avoidance → in abnormal scenarios, erroneous maps are automatically cleared without manual intervention, achieving closed-loop management and reuse of visual obstacle data.

[0088] In some instances, a lawnmower may include the following system modules: Perception module: Responsible for collecting visual data in real time and identifying static obstacles.

[0089] Navigation module: responsible for route planning, zone management, and abnormal status detection.

[0090] Map Management Module: Responsible for generating temporary maps, storing them permanently, loading and reusing them, and clearing them.

[0091] Storage module: Based on eMMC (Embedded Multi Media Card), persistent storage of area obstacle maps is implemented.

[0092] In some instances, such as Figure 8 As shown, the control method for a mobile cleaning device, specifically a lawnmower, may include the following steps S1 to S4: Step S1, First daytime operation: The lawnmower mows normally without any additional prompts, and obstacle detection and temporary mapping are completed silently in the background.

[0093] For example, the perception module continuously identifies static obstacles within the current work area and generates a temporary obstacle map. The navigation module tracks the work progress in real time and determines whether the current area has been completely cut.

[0094] Step S2, single zone operation completed: The lawnmower automatically saves the obstacle map of the area, which can be viewed in the log.

[0095] For example, after the navigation module determines that a single zone operation is complete, it sends a save command to the map management module. Based on the save command, the map management module serializes the temporary map and writes it to the eMMC storage.

[0096] In other words, the visual obstacle map for a single mowing area is only permanently stored after the mowing is completed, unlike existing technologies that store maps in real-time or during the process. Saving maps after each area is completed eliminates dynamic interference and temporary errors, resulting in more accurate map data and significantly improved obstacle avoidance stability. This avoids redundant mapping, reduces the computing power and energy consumption of mobile mowing equipment, speeds up operations, and minimizes the occurrence of lag and interruptions in mobile mowing equipment.

[0097] Step S3, Night / Secondary Operation: The lawnmower directly calls the saved offline map for obstacle avoidance, without relying on real-time visual recognition, and the obstacle avoidance performance is the same as during the day.

[0098] For example, when the lawnmower re-enters the area, the map management module loads a historical obstacle map from the eMMC. The navigation module plans obstacle avoidance paths directly based on the offline map, without needing to activate real-time visual perception.

[0099] In other words, a precise offline obstacle avoidance method can be achieved at night when there is no real-time visual perception, utilizing static obstacle history maps stored during the day, thus solving the pain point of low-light operations. It can create maps during the day and reuse them offline at night, completely resolving the problem of visual obstacle avoidance failure in low-light environments at night and extending the operating hours of lawnmowers.

[0100] Step S4, Abnormal Scenario: When the lawnmower cannot navigate / return to charging station, automatically clear the error map and re-find the route.

[0101] For example, when the navigation module detects that navigation is impossible or the device cannot recharge, it sends a clear map command to the map management module. The map management module deletes the currently erroneous map data based on the clear map command, and the navigation module replans the route.

[0102] In other words, when the lawnmower malfunctions and cannot navigate or return to its charging dock, it automatically clears the erroneous obstacle map to restore the normal path planning control strategy. Automatically clearing erroneous maps when navigation or recharging is abnormal ensures the mobile mower can autonomously extricate itself from trouble, reducing manual intervention.

[0103] Please see Figure 9 , Figure 9 This is a schematic block diagram of a control device provided in an embodiment of this application.

[0104] like Figure 9 As shown, the control device 300 includes a processor 301 and a memory 302, which are connected via a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0105] Specifically, processor 301 provides computing and control capabilities to support the operation of the entire control device 300. Processor 301 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0106] Specifically, the memory 302 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0107] The processor is used to run a computer program stored in the memory, and when executing the computer program, implements any of the control methods for the mobile cleaning device provided in the embodiments of this application.

[0108] In one embodiment, the processor 301 is configured to run a computer program stored in memory, and to perform the following steps when executing the computer program: The mobile cleaning equipment acquires path perception information as it moves through the work area; Based on the perceived information, the distribution of obstacles during the movement is confirmed, and an obstacle map of the work area is obtained based on the distribution of obstacles during the movement. In response to the retrieval command of the mobile cleaning device, the obstacle map is retrieved to control the mobile cleaning device to avoid obstacles during its movement.

[0109] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the control device described above can be referred to the corresponding process in the aforementioned control method embodiment of the mobile cleaning device, and will not be repeated here.

[0110] This application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the steps of any of the control methods for a mobile cleaning device provided in this application.

[0111] The storage medium can be the internal storage unit of the mobile cleaning device described in the foregoing embodiments, such as the hard drive or memory of the mobile cleaning device. Alternatively, the storage medium can be an external storage device of the mobile cleaning device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the mobile cleaning device.

[0112] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0113] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method for a mobile cleaning device, characterized in that, include: As the mobile cleaning device moves through the work area, it acquires path perception information. Based on the perceived information, the distribution of obstacles during the movement is confirmed, and an obstacle map of the work area is obtained based on the distribution of obstacles during the movement. In response to the retrieval command of the mobile cleaning device, the obstacle map is retrieved to control the mobile cleaning device to avoid obstacles during its movement.

2. The control method for the mobile cleaning device according to claim 1, characterized in that, After retrieving the obstacle map, the process also includes: Based on the path perception information acquired again by the mobile cleaning device, the distribution of new obstacles during the journey is confirmed; The obstacle map is updated based on the new obstacle distribution.

3. The control method for the mobile cleaning device according to claim 2, characterized in that, The obstacle map includes multiple sub-work area maps; updating the obstacle map based on the new obstacle distribution includes: Determine the sub-work area map corresponding to the new obstacle distribution; Update the corresponding sub-work area map based on the new obstacle distribution.

4. The control method for the mobile cleaning device according to claim 1, characterized in that, After retrieving the obstacle map, the process also includes: Based on the obstacle map, plan the recharge path of the mobile cleaning device; During the process of the mobile cleaning device traveling along the recharge path, in response to the confirmation that there are obstacles distributed along the recharge path, the obstacle map is updated and the recharge path is replanned.

5. The control method for the mobile cleaning device according to claim 1, characterized in that, The retrieval command is triggered based on the abnormal environmental conditions in which the mobile cleaning device is located; or The retrieval command is triggered based on an abnormal state of the sensing sensor used to acquire the sensing information.

6. The control method for the mobile cleaning device according to claim 5, characterized in that, The abnormal environmental conditions include at least one of the following: low light environment, strong light environment, and severe weather environment; The abnormal state includes at least one of the following: malfunction, obstruction by foreign objects, or unavailable sensing information.

7. The control method for the mobile cleaning device according to claim 1, characterized in that, After retrieving the obstacle map, the process also includes: In response to the detection of an error in the obstacle map, the obstacle map is deleted.

8. The control method for the mobile cleaning device according to claim 7, characterized in that, The detection of errors in the obstacle map includes: The mobile cleaning device acquires collision information with obstacles during its movement; or The mobile cleaning device acquires obstacles that are not recorded on the obstacle map during its movement.

9. The control method for the mobile cleaning device according to any one of claims 1-8, characterized in that, The work area is configured with multiple sub-work areas; The step of obtaining an obstacle map of the work area based on the distribution of obstacles during the movement includes: Based on the obstacle distribution during the movement of each sub-work area, a temporary obstacle map is generated for each sub-work area; Based on the temporary obstacle map of each of the sub-work areas, an obstacle map of the work area is obtained.

10. The control method for the mobile cleaning device according to claim 9, characterized in that, The process of obtaining an obstacle map for the work area based on a temporary obstacle map for each of the sub-work areas includes: After the work in each sub-work area is completed, a temporary obstacle map for each sub-work area is obtained and determined as a map for each sub-work area; or The temporary obstacle maps of each of the sub-work areas are merged to obtain the obstacle map of the work area.

11. The control method for the mobile cleaning device according to claim 9, characterized in that, The mobile cleaning device includes a lawnmower, and the area of ​​the sub-working area is less than or equal to the full-box mowing area of ​​the lawnmower. The full-box mowing area is used to characterize the mowing area corresponding to the grass collection capacity of the lawnmower's collection box.

12. A control device, characterized in that, The control device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the control method as described in any one of claims 1 to 11.

13. A mobile cleaning device, characterized in that, The mobile cleaning device includes a body, a sensing sensor, and a control device as described in claim 12, wherein the sensing sensor is used to acquire path perception information; the sensing sensor and the control device are disposed on the body.

14. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the control method as described in any one of claims 1 to 11.

Images