Cleaning device control method and apparatus, storage medium, controller, and device

Abstract

A cleaning device control method and apparatus, a storage medium, a controller, and a device. The method comprises: acquiring an image of a region to be cleaned (S11); performing image segmentation and edge processing on the image of said region (S12); constructing a target map of said region on the basis of the segmentation result and the edge processing result, the target map comprising obstacle information (S13); and controlling a cleaning device on the basis of the target map (S14).

Description

Control method, device, storage medium, controller, and equipment for cleaning equipment

[0001] Cross-references to related publications

[0002] This disclosure claims priority to Chinese patent application No. 202410322675.1 filed on March 20, 2024 and No. 202311278568.5 filed on September 28, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the technical field of cleaning equipment, and in particular to a control method, device, storage medium, controller, and equipment for cleaning equipment. Background Art

[0004] Currently, commercially available cleaning equipment, such as sweepers, primarily relies on laser instruments like LiDAR to gather information about the surrounding environment. These instruments then determine the location of obstacles and subsequently control cleaning accordingly. However, the information obtained by laser instruments is relatively limited. For example, it cannot accurately distinguish between uneven surfaces and low obstacles, nor can it distinguish between the type and location of stains. This prevents cleaning equipment from applying more complex cleaning strategies.

[0005] Public content

[0006] The present disclosure aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the purpose of the present disclosure is to provide a control method, device and storage medium, controller, and device for cleaning equipment to improve the working efficiency and accuracy of cleaning equipment in complex environments.

[0007] To achieve the above-mentioned objectives, the first aspect of the present disclosure proposes a control method for a cleaning device, comprising: acquiring an image of the area to be cleaned; performing image segmentation and edge processing on the image of the area to be cleaned; constructing a target map of the area to be cleaned based on the segmentation results and the edge processing results, wherein the target map includes obstacle information; and controlling the cleaning device based on the target map.

[0008] In addition, the control method of the cleaning device according to the embodiment of the present disclosure may also have the following additional technical features:

[0009] According to one embodiment of the present disclosure, a pre-trained segmentation model is used to perform image segmentation on the image of the area to be cleaned, and the training process of the segmentation model includes: acquiring a plurality of work scene images; dividing the plurality of work scene images into a training set and a test set, and respectively labeling the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include the background, the ground, and traversable obstacles on the ground; constructing a segmentation model, and pre-training the segmentation model using the training set and its corresponding annotation information, and testing the pre-trained segmentation model using the test set and its corresponding annotation information to obtain a final trained segmentation model.

[0010] According to one embodiment of the present disclosure, constructing a target map of the area to be cleaned based on the segmentation results and the edge processing results includes: performing corrosion processing on the ground area in the segmentation results to obtain a first image; obtaining an edge line where the background and the ground contact in the image of the area to be cleaned based on the edge processing results and the first image; and constructing the target map based on the edge line and traversable obstacles on the ground in the segmentation results.

[0011] According to one embodiment of the present disclosure, the image of the area to be cleaned is acquired by using an image acquisition element installed on the cleaning device, and the edge line where the background and the ground are in contact in the image of the area to be cleaned is obtained based on the edge processing result and the first image, including: removing the edge in the ground area in the edge processing result based on the first image; and determining the edge line in the edge processing result after removing the edge in the ground area according to the line of sight of the image acquisition element from near to far.

[0012] According to one embodiment of the present disclosure, constructing the target map based on the edge line and the traversable obstacles on the ground in the segmentation result includes: determining a traversable area based on the edge line, and obtaining the category and position of the traversable obstacles in the traversable area based on the traversable area and the traversable obstacles on the ground in the segmentation result; and constructing the target map based on the category and position of the traversable obstacles in the traversable area.

[0013] According to one embodiment of the present disclosure, controlling the cleaning device according to the target map includes: updating a global map according to the target map; determining a target cleaning area and a target cleaning strategy according to the updated global map; and controlling the cleaning device to clean the target cleaning area according to the target cleaning strategy.

[0014] To achieve the above-mentioned objectives, the second aspect embodiment of the present disclosure proposes a control device for cleaning equipment, including: an acquisition module for acquiring an image of the area to be cleaned; a processing module for performing image segmentation and edge processing on the image of the area to be cleaned; a construction module for constructing a target map of the area to be cleaned based on the edge processing results and the segmentation results; and a control module for controlling the cleaning equipment according to the target map.

[0015] To achieve the above objectives, the third embodiment of the present disclosure proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the control method of the cleaning device described in the first embodiment is implemented.

[0016] To achieve the above-mentioned objectives, the fourth embodiment of the present disclosure proposes a controller, comprising a memory, a processor and a computer program stored on the memory. When the computer program is executed by the processor, the control method of the cleaning equipment described in the first embodiment above is implemented.

[0017] To achieve the above-mentioned objectives, a fifth embodiment of the present disclosure proposes a cleaning device, including the above-mentioned controller.

[0018] According to the control method, apparatus, storage medium, controller, and device for cleaning equipment of the disclosed embodiments, an image of the area to be cleaned is first acquired; the image of the area to be cleaned is then segmented and edge processed; a target map of the area to be cleaned, including obstacle information, is constructed based on the segmentation and edge processing results; and finally, the cleaning equipment is controlled based on the target map. Thus, by fusing the image segmentation results with the edge processing results to construct a target map for controlling the cleaning equipment, the efficiency and accuracy of the cleaning equipment in complex environments are improved.

[0019] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present disclosure;

[0021] FIG2 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present disclosure;

[0022] FIG3 is a training flowchart of a segmentation model according to an embodiment of the present disclosure;

[0023] FIG4 is a flowchart of constructing a target map according to a specific embodiment of the present disclosure;

[0024] FIG5( a ) is an image of an area to be cleaned according to an example of the present disclosure;

[0025] Figure 5(b) is the segmentation result of the image in Figure 4(a);

[0026] Figure 5(c) is a schematic diagram of the edge line obtained based on Figure 4(a);

[0027] Figure 5(d) is the target map constructed based on Figure 4(a);

[0028] FIG6 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present disclosure;

[0029] FIG7 is a schematic diagram of a grid map provided by an embodiment of the present disclosure;

[0030] FIG8 is a schematic diagram of an updated grid map provided by an embodiment of the present disclosure;

[0031] FIG9 is a schematic diagram of an obstacle avoidance trajectory of a sweeping robot provided by an embodiment of the present disclosure;

[0032] FIG10 is a schematic diagram of marking a circle track in an obstacle avoidance track of a sweeping robot according to an embodiment of the present disclosure;

[0033] FIG11 is a schematic flow chart of another method for controlling a cleaning device according to an embodiment of the present disclosure;

[0034] FIG12 is a flow chart of another method for controlling a cleaning device according to an embodiment of the present disclosure;

[0035] FIG13 is a schematic structural diagram of a control device for a cleaning device according to an embodiment of the present disclosure;

[0036] FIG14 is a structural block diagram of a controller according to an embodiment of the present disclosure;

[0037] FIG15 is a structural block diagram of a cleaning device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0039] The following describes the control method, device, storage medium, controller, and device of the cleaning device according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0040] FIG1 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present disclosure.

[0041] As shown in FIG1 , the control method of the cleaning equipment includes:

[0042] S1, obtaining data of the area to be cleaned.

[0043] S2, processing the data of the area to be cleaned.

[0044] S3: Based on the data processing results, a target map of the area to be cleaned is constructed, where the target map includes obstacle information.

[0045] S4, controlling the cleaning equipment according to the target map.

[0046] In some embodiments, the data of the area to be cleaned may include: an image of the area to be cleaned, lidar data, or other sensor data. The following detailed description will be given by taking the cleaning area data as an image of the area to be cleaned as an example.

[0047] FIG2 is a flow chart of a method for controlling a cleaning device according to an embodiment of the present disclosure. As shown in FIG2 , the method for controlling a cleaning device includes:

[0048] S11, acquiring an image of the area to be cleaned.

[0049] The image of the area to be cleaned may be acquired by using an image acquisition element installed on the cleaning device (such as a monocular camera installed in front of the cleaning device).

[0050] Specifically, taking the acquisition of images by a monocular camera as an example, the image of the area to be cleaned can be collected by the monocular camera, and the collected image of the area to be cleaned can be stored in the memory equipped with the cleaning equipment for easy retrieval, wherein the image of the area to be cleaned can be an RGB image. The monocular camera is installed in front of the cleaning equipment and can effectively observe the ground area within a certain range. Compared with the scene information obtained by laser instruments such as lidar, the monocular camera can obtain richer scene information, which is convenient for subsequent more accurate judgment of low obstacles and allows the cleaning equipment to adopt a more optimized cleaning plan based on the obstacle category information. In addition, the monocular camera has a simple structure and low cost, and the images obtained are easy to calibrate and identify.

[0051] S12, performing image segmentation and edge processing on the image of the area to be cleaned.

[0052] Specifically, the canny edge processing algorithm, the sobel edge processing algorithm, etc. may be used to perform edge processing on the image of the area to be cleaned.

[0053] The Canny algorithm is a classic edge processing algorithm. The processing process includes: first, applying a Gaussian blur to the image to suppress high-frequency noise; then, using the Sobel operator to calculate the gradient magnitude and direction; then, performing non-maximum pixel gradient suppression on the image to eliminate spurious responses caused by edge processing. The basic method is to compare the gradient strength of the current pixel with the gradient strength of adjacent pixels along the positive and negative gradient directions. If the gradient strength is an extreme value, the pixel is retained as an edge point; if not, it is suppressed and not considered an edge point. Then, threshold hysteresis is performed on the image, defining a high threshold and a low threshold. Pixels with gradient strengths below the low threshold are suppressed and not considered edge points. Pixels with gradient strengths above the high threshold are defined as strong edges and retained as edge points. Pixels between the high and low thresholds are defined as weak edges and reserved for further processing. Finally, isolated weak edge suppression is performed on the image. For each weak edge in the previous step, if one of the eight adjacent pixels is a strong edge, the weak edge is considered a strong edge; otherwise, it is considered an isolated point and discarded.

[0054] In some embodiments of the present disclosure, a pre-trained segmentation model is used to perform image segmentation on the image of the area to be cleaned, and the training process of the segmentation model includes: acquiring a plurality of work scene images; dividing the plurality of work scene images into a training set and a test set, and respectively labeling the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include the background, the ground, and traversable obstacles on the ground (such as carpets, stains, etc.); constructing a segmentation model, and pre-training the segmentation model using the training set and its corresponding annotation information, and testing the pre-trained segmentation model using the test set and its corresponding annotation information to obtain a final trained segmentation model.

[0055] Specifically, considering the accuracy and lightness comprehensively, the DeepLab v3+ segmentation model can be used for image segmentation, and the segmentation model training process is shown in Figure 3. First, images of work scenes (such as home scenes) are collected, and the collected images are screened to ensure the quantitative balance between categories and the correct division of training sets and test sets to a certain extent. Then, the screened images are labeled with categories (that is, the targets to be inspected are marked). Among them, in order to ensure the diversity of targets to be inspected and work scenes, the collected images can cover a variety of household types, multiple target types, multiple ambient lighting, multiple shooting distances and multiple shooting angles, etc. In addition, in order to improve the detection efficiency, the screened images can also be preprocessed, such as image enhancement processing, cropping processing, denoising processing, etc.

[0056] Next, select or build an image segmentation model, such as the DeepLab v3+ segmentation model mentioned above. Next, set up a model training environment and train the image segmentation model using the training set. Then, evaluate and optimize the image segmentation model using the test set. Repeat this process to complete the image segmentation model training and obtain the segmentation model required for cleaning equipment, which is the final trained segmentation model.

[0057] DeepLab v3+ is a semantic segmentation algorithm that assigns a category to each pixel in an image, but does not distinguish between objects in the same category. Compared to traditional semantic segmentation algorithms, DeepLab v3+'s most significant feature is the introduction of dilated convolutions, which extract more effective image features without losing information, ensuring that each convolution output contains a wider range of information.

[0058] It should be noted that the above-mentioned image segmentation and edge processing can be implemented by one integrated processing module or by two separate processing modules; they can be performed simultaneously or sequentially.

[0059] S13: Constructing a target map of the area to be cleaned based on the segmentation results and edge processing results.

[0060] The target map may include obstacle information.

[0061] In some embodiments of the present disclosure, a target map of the area to be cleaned is constructed based on the segmentation results and the edge processing results, including: performing corrosion processing on the ground area in the segmentation results to obtain a first image; obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned based on the edge processing results and the first image; and constructing a target map based on the edge line and traversable obstacles on the ground in the segmentation results.

[0062] Erosion is a basic morphological operation that targets highlights in an image. This operation erodes highlights from the original image, creating a smaller area. The process involves covering each pixel in the original image with the center of a structuring element and replacing the pixel value covered by the center of the structuring element with the minimum value of the covered pixels.

[0063] Specifically, the segmentation model can preliminarily segment the field of view area in front of the cleaning equipment, but due to labeling errors and inference errors, it is still impossible to obtain a more accurate area edge. To this end, the present disclosure performs corrosion processing on the ground area in the image segmentation result to obtain a first image to expose more edge areas and avoid the ground segmentation result covering the edge. Then, the edge processing result can be combined with the edge processing result to obtain the edge of the ground area without the edge, and then a more accurate contact edge of the non-crossable obstacle in the background with the ground can be obtained to obtain the corresponding edge line. Afterwards, based on the edge line and the surmountable obstacles on the ground in the segmentation result, a target map can be constructed, which can show the cleanable area of ​​the cleaning equipment, the category of the surmountable obstacles in the cleanable area, etc.

[0064] In some embodiments of the present disclosure, an edge line where the background and the ground meet in the image of the area to be cleaned is obtained based on the edge processing result and the first image, including: removing the edges in the ground area in the edge processing result based on the first image; and determining the edge line in the edge processing result after removing the edges in the ground area according to the line of sight of the image acquisition element from near to far.

[0065] Specifically, the edge processing result can be a binary edge map. Based on the first image, the edges in the ground area of ​​the binary edge map can be removed. Subsequently, the binary edge map, after removing the edges in the ground area, can be searched from near to far along the line of sight of the image acquisition element located at the front end of the cleaning device until the edge pixel location is found. This process includes searching each column of the binary edge map after removing the edges in the ground area, starting from the point closest to the cleaning device and moving toward the farther point. The first non-zero pixel found is the edge pixel. The line connecting the found edge pixels is the aforementioned edge line.

[0066] In some embodiments of the present disclosure, a target map is constructed based on edge lines and traversable obstacles on the ground in segmentation results, including: determining a traversable area based on the edge lines, and obtaining the category and position of the traversable obstacles in the traversable area based on the traversable area and the traversable obstacles on the ground in the segmentation results; and constructing a target map based on the category and position of the traversable obstacles in the traversable area.

[0067] Specifically, the area where the edge line in the image of the area to be cleaned is close to the image acquisition element side is the passable area, and then the category and position of the passable obstacles in the passable area are obtained based on the passable area and the passable obstacles on the ground in the segmentation result (such as carpets, stains, etc.). Afterwards, the coordinates of the passable obstacles in the passable area are converted from the image coordinate system to the world coordinate system to construct a target map. When performing coordinate conversion, the first conversion matrix between the image coordinate system and the camera coordinate system can be used to convert the coordinates of the passable obstacles from the image coordinate system to the camera coordinate system, and then the second conversion matrix between the camera coordinate system and the world coordinate system can be used to convert the coordinates of the passable obstacles from the camera coordinate system to the world coordinate system. Among them, the first conversion matrix can be obtained based on the intrinsic parameters, extrinsic parameters, center, distortion, etc. of the image acquisition element, and the second conversion matrix can be obtained based on the camera posture calculated by the lidar.

[0068] S14, controlling the cleaning equipment according to the target map.

[0069] In some embodiments of the present disclosure, controlling the cleaning equipment according to the target map includes: updating the global map according to the target map; determining the target cleaning area and the target cleaning strategy according to the updated global map; and controlling the cleaning equipment to clean the target cleaning area according to the target cleaning strategy.

[0070] Specifically, the first global map can be custom-built or obtained by fusing multiple target maps corresponding to the entire work scene. The cleaning equipment can navigate according to the global map, such as from the bedroom to the kitchen. At the same time, the above steps S11-S13 can be used to build a target map during navigation, and the global map can be updated according to the target map. Afterwards, the target cleaning area (such as the currently uncleaned and passable area) and the target cleaning strategy (such as avoiding carpets and stains during navigation; lifting the mop on the carpet to increase suction; focusing on stains; avoiding obstacles that cannot be identified by sensors such as lidar and line lasers) can be determined according to the updated global map, and the cleaning equipment can be controlled to clean the target cleaning area according to the target cleaning strategy. In this way, the working efficiency and accuracy of the cleaning equipment can be improved.

[0071] For ease of understanding, the control method of the cleaning device of the present disclosure is described below in conjunction with the specific embodiment process shown in FIG4 and the specific scenario examples shown in FIG5(a) to FIG5(d):

[0072] As shown in Figure 4, an image is first acquired, resulting in an image of the area to be cleaned, as shown in Figure 5(a). Then, the image segmentation model is used to segment the image of the area to be cleaned, resulting in the segmentation result shown in Figure 5(b). Region B represents the ground, region A represents the carpet, and the unlabeled portion represents the background (including insurmountable obstacles). The ground region in Figure 5(b) is then eroded to expose more edge regions and prevent the ground segmentation result from overlapping the edges. Furthermore, the Canny edge processing algorithm is used to process the edges of the image of the area to be cleaned, resulting in an edge processing result.

[0073] Afterwards, the edge of the ground area in the edge processing result is removed by combining the corrosion processing result, and the edge pixel positions of the remaining edges in the edge processing result are found from near to far along the line of sight of the image acquisition component to obtain the edge line of the removed ground area, as shown by the thick white line in Figure 5(c).

[0074] Next, determine the pixel coordinates of the edge where the obstacle and the ground meet, and determine the obstacle category. The ground area before reaching the edge line is regarded as a passable area (i.e., the ground without insurmountable obstacles). The segmentation result of the segmentation model already contains the category of surmountable obstacles and the coordinates in the image coordinate system. By querying the calibration file (including the transformation matrix), the coordinates of the surmountable obstacles in the image coordinate system are converted to the world coordinate system, as shown in Figure 5(d). The figure shows the carpet position (i.e., the black block part in Figure 5(d)) and the position of the insurmountable obstacle (i.e., the line part in Figure 5(d)). Thus, a BEV (Bird's Eye View), i.e., the target map, is constructed. Afterwards, the target cleaning area and target cleaning strategy can be further determined in combination with the global map, and the cleaning equipment can be controlled to perform cleaning work according to the target cleaning area and target cleaning strategy.

[0075] Among them, a bird's-eye view is a perspective of viewing an object or scene from above. In the field of obstacle avoidance, data obtained by sensors (such as image acquisition components) are usually converted into BEV representations to better perform tasks such as object detection and path planning. The advantages of BEV include: being able to simplify complex three-dimensional environments into two-dimensional images, which can save a lot of resources in computing and storage; providing a unique visual effect that makes the objects and spatial relationships in the scene more clearly visible; processing tasks such as object detection, tracking, and classification in BEV is much simpler than processing directly in the original 3D data; image acquisition component detection will result in larger objects when closer and smaller objects when farther away, while BEV has almost no scale differences among similar targets, making it easier to learn feature scale consistency.

[0076] The present disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0077] An embodiment of the present disclosure provides a method for setting an obstacle avoidance trajectory, as shown in FIG6 , the method includes the following steps:

[0078] Step 101: Determine a first obstacle avoidance trajectory of a mobile device.

[0079] It is understandable that mobile devices include but are not limited to robot cleaners, window cleaning robots, and campus delivery robots, and this disclosure does not limit this.

[0080] In the disclosed embodiments, the mobile device is a sweeping robot as an example. The sweeping robot is also called an automatic cleaning machine, smart vacuum cleaner, robot vacuum cleaner, etc. It is a kind of smart home appliance that can automatically complete the floor cleaning work in the room. Generally, it adopts the brushing and vacuuming method to first absorb the debris on the ground into its own garbage collection box, thereby completing the function of floor cleaning. Generally speaking, robots that complete the work of sweeping, vacuuming, and mopping are also collectively classified as sweeping robots. The body of the sweeping robot is a movable device of automation technology, and a vacuum cleaner with a dust collection box cooperates with the body to set the control path, and walks repeatedly in the room, such as cleaning along the edge, centralized cleaning, random cleaning, straight line cleaning and other paths, and is supplemented by side brushes, central main brush rotation, rags and other methods to enhance the cleaning effect, so as to complete the anthropomorphic home cleaning effect.

[0081] A sweeping robot is usually equipped with multiple types of sensors, which may include lidar, camera, line laser, position sensitive detector (PSD), collision plate and inertial measurement unit (IMU); lidar can measure the distance and angle of obstacles around the robot; line laser and PSD can detect obstacles and edges on the ground; collision plate can detect collisions near the robot; IMU can detect the position and posture of the robot; the data collected by the sensors is processed to construct a map of the environment in which the sweeping robot is located; for example, the collected sensor data can be used to identify obstacles through artificial intelligence (AI), the obstacles in the environment in which the sweeping robot is located can be classified, and the obstacle information can be integrated into the probability grid map of the simultaneous localization and mapping (SLAM) system to generate an environment map with obstacle markings, and the navigation trajectory of the sweeping robot can be set according to the environment map so that the sweeping robot can avoid obstacles.

[0082] The obstacles in the environment of the sweeping robot will change, so the navigation trajectory of the sweeping robot will also change with the changes in the obstacles. The first obstacle avoidance trajectory can be a navigation trajectory set for the sweeping robot based on the marked first obstacle. The first obstacle can include but is not limited to obstacles with large volume and relatively fixed position; for example: walls, cabinets, refrigerators and water dispensers, etc.

[0083] In practical applications, determining the first obstacle-avoiding trajectory of the mobile device can be understood as determining a navigation trajectory of the cleaning robot to avoid the first obstacle based on the environment map.

[0084] Step 102: If the first obstacle avoidance trajectory meets the update condition, the first obstacle avoidance trajectory is updated to obtain a second obstacle avoidance trajectory; wherein the second obstacle included in the second obstacle avoidance trajectory has different object characteristics from the first obstacle included in the first obstacle avoidance trajectory.

[0085] It is understood that the update conditions can be set based on actual circumstances, and this disclosure does not specifically limit this. For example, the actual circumstances may include the robot vacuuming detecting a second obstacle while cleaning along a first obstacle avoidance trajectory. The actual circumstances may also include adding the second obstacle to the first obstacle avoidance trajectory, for example, by displaying the first obstacle avoidance trajectory on a human-computer interaction interface and adding the second obstacle to the first obstacle avoidance trajectory.

[0086] The second obstacle may be a small physical obstacle with a flexible position, which is not limited in the present disclosure. For example, the second obstacle may be a columnar physical obstacle, such as a table leg.

[0087] In the embodiment of the present disclosure, when the sweeping robot is moving along the first obstacle avoidance trajectory, a second obstacle is detected and the second obstacle can be marked in the environment map; or the second obstacle can be added to the first obstacle avoidance trajectory through the human-computer interaction interface to update the first obstacle avoidance trajectory and obtain the second obstacle avoidance trajectory; Figure 7 is a grid map constructed according to the environment in which the sweeping robot is located, and Figure 8 is a grid map after the second obstacle is marked on the basis of Figure 7.

[0088] Step 103: Control the mobile device to move along the second obstacle avoidance trajectory, so that the mobile device avoids the first obstacle and the second obstacle during the movement.

[0089] In the embodiment of the present disclosure, controlling the mobile device to move along the second obstacle avoidance trajectory can be understood as controlling the sweeping robot to move along the second obstacle avoidance trajectory, avoiding the first obstacle and the second obstacle in the process of completing the cleaning process.

[0090] From the above content, it can be seen that the obstacle avoidance trajectory setting method provided by the present disclosure determines a first obstacle avoidance trajectory of a mobile device. If the first obstacle avoidance trajectory meets the update condition, the first obstacle avoidance trajectory is updated to obtain a second obstacle avoidance trajectory, so that the mobile device can avoid the first obstacle and the second obstacle during movement. This solves the problem in the related art that indoor mobile robots can only move according to the set obstacle avoidance trajectory and have a single obstacle avoidance method, makes the environmental map more consistent with the real environment, and realizes the autonomous navigation and safe obstacle avoidance of indoor mobile robots in complex environments.

[0091] In some embodiments of the present disclosure, if the first obstacle avoidance trajectory satisfies the update condition, step 102 updates the first obstacle avoidance trajectory to obtain the second obstacle avoidance trajectory, which can be achieved by the following steps:

[0092] Controlling the mobile device to move along a first obstacle avoidance trajectory so that the mobile device avoids the first obstacle during movement;

[0093] If a second obstacle exists while the mobile device moves along the first obstacle avoidance trajectory, it is determined that the first obstacle avoidance trajectory meets an update condition, and the first obstacle avoidance trajectory is updated based on the second obstacle to obtain a second obstacle avoidance trajectory.

[0094] In the disclosed embodiment, when the sweeping robot moves along the first obstacle avoidance trajectory, the sweeping robot will automatically avoid the first obstacle. During this process, the sweeping robot can detect obstacles through the multiple types of sensors configured on itself. If a second obstacle is detected, the first obstacle avoidance trajectory is updated to obtain the second obstacle avoidance trajectory.

[0095] In some embodiments of the present disclosure, the first obstacle includes an obstacle determined based on first sensing data of a first sensor, and / or the second obstacle includes an obstacle determined based on second sensing data of a second sensor.

[0096] In the embodiment of the present disclosure, the first sensor may be a lidar; the first sensing data may include obstacle data detected by the lidar of the sweeping robot; the second sensor may be a sensor other than the lidar, such as: line laser, PSD, collision plate and camera, etc.; the second sensing data may include obstacle data detected by sensors other than radar laser of the sweeping robot.

[0097] In some embodiments of the present disclosure, when controlling the mobile device to move along the first obstacle avoidance trajectory, the method further includes:

[0098] Obtaining obstacle avoidance data of the mobile device during the movement of the mobile device along the first obstacle avoidance trajectory;

[0099] If the obstacle avoidance data satisfies the obstacle avoidance condition, it is determined that the obstacle avoidance object corresponding to the obstacle avoidance data is the second obstacle.

[0100] It is understood that obstacle avoidance data refers to data related to obstacles avoided by the robot vacuum during movement, which can be collected by the robot vacuum's own multiple sensors. Obstacle avoidance targets are obstacles that the robot vacuum avoids during movement. Obstacle avoidance conditions can be set based on actual needs and are not limited in this disclosure. For example, the actual need can be the type of obstacle to be avoided.

[0101] In the embodiment of the present disclosure, obtaining the obstacle avoidance data of the mobile device during the movement of the mobile device along the first obstacle avoidance trajectory can be understood as obtaining the obstacle avoidance data collected by the sweeping robot during the movement along the first obstacle avoidance trajectory, processing the obstacle avoidance data, confirming the type of obstacles encountered by the sweeping robot in this process, and if there is an obstacle type that meets the obstacle avoidance condition, determining that the obstacle that meets the obstacle avoidance condition is the second obstacle.

[0102] In some embodiments of the present disclosure, the obstacle avoidance data includes posture data, and the obstacle avoidance data satisfies obstacle avoidance conditions, including:

[0103] determining a yaw trajectory of the mobile device during movement along a first obstacle avoidance trajectory based on the posture data;

[0104] If the yaw trajectory meets the trajectory characteristics, it is determined that the obstacle avoidance data meets the obstacle avoidance conditions.

[0105] It is understood that posture data includes but is not limited to the robot's movement direction and coordinate position; the yaw trajectory can be understood as the robot's movement trajectory as it deviates from the first obstacle avoidance trajectory; the trajectory characteristics can be set according to actual conditions; for example, the trajectory characteristic can be a circular trajectory, i.e., a circular trajectory. Specifically, the Hough circle detection algorithm can be used to identify the robot's circular trajectory during movement. Figure 9 is a schematic diagram of the robot's movement trajectory along the first obstacle avoidance trajectory, and Figure 10 is a schematic diagram based on Figure 9 with the circular trajectory marked.

[0106] In the embodiment of the present disclosure, determining the yaw trajectory of the mobile device during its movement along the first obstacle avoidance trajectory based on the posture data can be understood as determining the yaw trajectory that deviates from the first obstacle avoidance trajectory during the movement based on the moving direction and coordinate position of the sweeping robot. If the yaw trajectory is a circular trajectory, determining that the obstacle corresponding to the circular trajectory meets the obstacle avoidance condition.

[0107] In some embodiments of the present disclosure, the obstacle avoidance data includes collision data, and the obstacle avoidance data satisfies obstacle avoidance conditions, including:

[0108] Determining a collision angle and a collision intensity of the mobile device during movement along the first obstacle avoidance trajectory based on the collision data;

[0109] If the collision angle meets the preset angle and the collision intensity meets the preset intensity, it is determined that the obstacle avoidance data meets the obstacle avoidance conditions.

[0110] It is understood that the collision data is the data collected by the collision plate during the movement of the sweeping robot; the preset angle and preset force can be set according to the actual situation, and the present disclosure does not limit this. The actual situation can be the hardness of the obstacle material and the curvature of the obstacle surface.

[0111] In the embodiment of the present disclosure, the angle and intensity of the collision of the sweeping robot with different obstacles during movement are determined based on the data collected by the collision plate, and it is determined whether there is an obstacle whose collision angle meets the preset angle and collision intensity meets the preset strength. If so, it is determined that the obstacle meets the obstacle avoidance conditions.

[0112] In some embodiments of the present disclosure, the obstacle avoidance data includes the area and outline corresponding to the obstacle, and the obstacle avoidance data satisfies the obstacle avoidance conditions, including:

[0113] If the area corresponding to the obstacle meets the preset area and the outline meets the preset shape, it is determined that the obstacle avoidance data meets the obstacle avoidance condition.

[0114] It is understandable that the area and outline of the obstacle can be calculated based on the trajectory of the sweeping robot to avoid the obstacle; the preset shape and preset area can be set according to the type of obstacle to be marked, and this disclosure does not limit this. For example, the preset shape can be a circle and the preset area can be 100cm 2 .

[0115] In the embodiment of the present disclosure, it is calculated based on the trajectory of the sweeping robot during movement whether there is an obstacle that meets the preset area and the outline meets the preset shape. If so, it is determined that the obstacle meets the obstacle avoidance condition.

[0116] In some embodiments of the present disclosure, when controlling the mobile device to move along the first obstacle avoidance trajectory, the method further includes:

[0117] Obtaining the area of ​​obstacles avoided by the mobile device during movement along the first obstacle avoidance trajectory;

[0118] If the area is smaller than the preset threshold, it is determined that a second obstacle exists.

[0119] It is understandable that the second obstacle is a columnar entity obstacle, and the preset threshold can be set according to the columnar entity obstacle, for example, the preset area is set to 10cm 2 .

[0120] In the embodiment of the present disclosure, the areas corresponding to the obstacles avoided by the mobile device during movement along the first obstacle avoidance trajectory are calculated to determine whether there is an obstacle with a corresponding area smaller than a preset threshold. If so, the obstacle is determined to be a columnar entity obstacle.

[0121] In some embodiments of the present disclosure, step 101 of determining the first obstacle avoidance trajectory of the mobile device may be implemented by the following steps:

[0122] Obtaining first obstacle information;

[0123] An environment map is constructed based on the first obstacle information, and a first obstacle avoidance trajectory is generated.

[0124] In the disclosed embodiment, the first obstacle information is collected by the sensor of the sweeping robot, the first obstacle information is integrated into the probability grid map of the same SLAM system, an environment map marked with the first obstacle is generated, and the first obstacle avoidance trajectory of the sweeping robot is set according to the environment map.

[0125] In some embodiments of the present disclosure, if the first obstacle avoidance trajectory satisfies the update condition, step 102 of updating the first obstacle avoidance trajectory to obtain the second obstacle avoidance trajectory may also be implemented by the following steps:

[0126] Displaying the first obstacle avoidance trajectory on the human-computer interaction interface;

[0127] When the mobile device moves along the first obstacle avoidance trajectory, a second obstacle marked for the first obstacle avoidance trajectory is received on the human-computer interaction interface, and it is determined that the first obstacle avoidance trajectory meets the update condition;

[0128] The first obstacle avoidance trajectory is updated based on the second obstacle to obtain a second obstacle avoidance trajectory, and the second obstacle avoidance trajectory is displayed on the human-computer interaction interface.

[0129] In the embodiments of the present disclosure, the human-computer interaction interface may include, but is not limited to, a mobile phone interface and a tablet interface, for example, an interaction interface of an application on a mobile phone or tablet.

[0130] In the embodiment of the present disclosure, the first obstacle avoidance trajectory of the sweeping robot can be displayed on the interactive interface of the mobile phone application. When the mobile device moves along the first obstacle avoidance trajectory, the marking information of the second obstacle can be received through the mobile phone application. After receiving the marking information, the first obstacle avoidance trajectory is updated to obtain the second obstacle avoidance trajectory. The second obstacle avoidance trajectory of the sweeping robot is displayed on the interactive interface of the mobile phone application to update the obstacle avoidance trajectory of the sweeping robot in real time.

[0131] In a feasible scenario, as shown in FIG11 , the setting of the obstacle avoidance trajectory of the sweeping robot can be achieved by the following steps:

[0132] S601: Build an indoor mobile robot map through SLAM.

[0133] S602: Combines collision plate, lidar, line laser, PSD sensor and AI obstacle recognition to set the navigation trajectory of the sweeping robot.

[0134] S603: Obtain the robot's position and posture when avoiding obstacles, and obtain the obstacle avoidance trajectory.

[0135] S604: Obstacle avoidance centers of several obstacle avoidance trajectories are obtained using graphics algorithms such as binarization, Gaussian blur, and Hough circle detection.

[0136] S605: Draw an obstacle mark at the center of the obstacle avoidance, and display the map, track, and obstacle on the APP to avoid the track bypassing the empty space.

[0137] An embodiment of the present disclosure provides a method for setting an obstacle avoidance trajectory. As shown in FIG12 , the method includes the following steps:

[0138] S701: Acquire a first map.

[0139] In the embodiment of the present disclosure, the first map may be any map that can mark obstacles; for example, a grid map constructed by a SLAM system; the first map may include the first obstacle that has been identified by the sweeping robot.

[0140] S702: Obtain obstacle information when the mobile device moves according to the first map.

[0141] In the disclosed embodiment, the sweeping robot can obtain different obstacle information based on the various sensors provided on the robot during movement.

[0142] S703: Update the obstacles on the first map according to the obstacle information, and display the updated obstacles.

[0143] In the disclosed embodiment, the sweeping robot can determine obstacles different from the first obstacle based on obstacle information collected by the sensor during movement, thereby updating the obstacles on the first map; the updated obstacles can be displayed on the first map through the human-computer interaction interface.

[0144] In some embodiments of the present disclosure, the updated obstacle includes a first obstacle before the update and a newly added second obstacle, and the second obstacle has different object features from the first obstacle.

[0145] In the embodiment of the present disclosure, the sweeping robot determines different categories of obstacles according to obstacle information during movement, thereby distinguishing a second obstacle different from the first obstacle, and updates the second obstacle on the first map.

[0146] In some embodiments of the present disclosure, step S703 of updating obstacles on the first map according to the obstacle information may be implemented by the following steps:

[0147] If the obstacle information indicates that a second obstacle exists when the mobile device moves according to the first map, the obstacle on the first map is updated.

[0148] In the embodiment of the present disclosure, the method for determining the second obstacle based on the obstacle information is the same as the above method and will not be described in detail here.

[0149] In some embodiments of the present disclosure, the first obstacle includes an obstacle determined based on first sensing data of a first sensor, and / or the second obstacle includes an obstacle determined based on second sensing data of a second sensor.

[0150] In the embodiment of the present disclosure, the first sensor may be a lidar; the first sensing data may include obstacle data detected by the lidar of the sweeping robot; the second sensor may be a sensor other than the lidar, such as: line laser, PSD, collision plate and camera, etc.; the second sensing data may include obstacle data detected by sensors other than radar laser of the sweeping robot.

[0151] In some embodiments of the present disclosure, step S702 of obtaining obstacle information when the mobile device moves according to the first map can be implemented by the following steps:

[0152] Displaying the first obstacle on the human-computer interaction interface;

[0153] When the mobile device moves along the first map, a marked second obstacle is received on the human-computer interaction interface to obtain obstacle information.

[0154] In the embodiment of the present disclosure, the method of displaying the first obstacle and receiving obstacle information is the same as the above method and will not be described in detail here.

[0155] In some embodiments of the present disclosure, after step S703 updates the obstacles on the first map according to the obstacle information, the method includes:

[0156] The mobile device is controlled to avoid a first obstacle and a second obstacle during movement.

[0157] In the embodiment of the present disclosure, the sweeping robot can avoid the first obstacle and the second obstacle during movement according to the map markings.

[0158] Corresponding to the target detection method of the above embodiment, the present disclosure also proposes a control device for cleaning equipment.

[0159] FIG13 is a schematic structural diagram of a control device of a cleaning device according to an embodiment of the present disclosure.

[0160] As shown in FIG13 , the control device 500 of the cleaning equipment includes: an acquisition module 501 , a detection module 502 , a segmentation module 503 , a construction module 504 and a control module 505 .

[0161] Among them, the acquisition module 501 is used to acquire the image of the area to be cleaned; the processing module 502 is used to perform image segmentation and edge processing on the image of the area to be cleaned; the construction module 504 is used to construct a target map of the area to be cleaned based on the edge processing results and the segmentation results, and the target map includes obstacle information; the control module 505 is used to control the cleaning equipment according to the target map.

[0162] In some embodiments of the present disclosure, a pre-trained segmentation model is used to perform image segmentation on the image of the area to be cleaned, and the training process of the segmentation model includes: acquiring a plurality of work scene images; dividing the plurality of work scene images into a training set and a test set, and respectively labeling the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include the background, the ground, and traversable obstacles on the ground; constructing a segmentation model, and pre-training the segmentation model using the training set and its corresponding annotation information, and testing the pre-trained segmentation model using the test set and its corresponding annotation information to obtain a final trained segmentation model.

[0163] In some embodiments of the present disclosure, a target map of the area to be cleaned is constructed based on the edge processing results and the segmentation results, including: performing corrosion processing on the ground area in the segmentation results to obtain a first image; obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned based on the edge processing results and the first image; and constructing a target map based on the edge line and traversable obstacles on the ground in the segmentation results.

[0164] In some embodiments of the present disclosure, the image of the area to be cleaned is acquired by using an image acquisition element installed on the cleaning equipment, and the edge line where the background and the ground meet in the image of the area to be cleaned is obtained based on the edge processing result and the first image, including: based on the edge of the background and the ground in the edge processing result, the edge line is determined in the first image from near to far according to the line of sight of the image acquisition element.

[0165] In some embodiments of the present disclosure, a target map is constructed based on edge lines and traversable obstacles on the ground in segmentation results, including: determining a traversable area based on the edge lines, and obtaining the category and position of the traversable obstacles in the traversable area based on the traversable area and the traversable obstacles on the ground in the segmentation results; and constructing a target map based on the category and position of the traversable obstacles in the traversable area.

[0166] In some embodiments of the present disclosure, controlling the cleaning equipment according to the target map includes: updating the global map according to the target map; determining the target cleaning area and the target cleaning strategy according to the updated global map; and controlling the cleaning equipment to clean the target cleaning area according to the target cleaning strategy.

[0167] It should be noted that, for other specific implementations of the control device 500 of the cleaning equipment in the embodiment of the present disclosure, reference may be made to the specific implementations of the control method of the cleaning equipment in the above embodiment of the present disclosure.

[0168] Based on the control method of the cleaning device in the above embodiment, the present disclosure also proposes a computer-readable storage medium.

[0169] In this embodiment, a computer program is stored thereon, and when the computer program is executed by the processor, the above-mentioned control method of the cleaning device is implemented.

[0170] Based on the control method of the cleaning device in the above embodiment, the present disclosure also proposes a controller.

[0171] FIG14 is a structural block diagram of a controller according to an embodiment of the present disclosure.

[0172] As shown in the figure, the controller 600 includes a processor 601, a memory 603 and a computer program stored in the memory. When the computer program is executed by the processor, the above-mentioned control method of the cleaning device is implemented.

[0173] The processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. The processor 601 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0174] Bus 602 may include a path for transmitting information between the aforementioned components. Bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, for example. Bus 602 may be divided into an address bus, a data bus, a control bus, and so on. For ease of illustration, FIG14 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0175] The memory 603 is used to store a computer program corresponding to the control method of the cleaning device of the above embodiment of the present disclosure, and the computer program is controlled and executed by the processor 601. The processor 601 is used to execute the computer program stored in the memory 603 to implement the content shown in the above method embodiment.

[0176] The controller 600 includes, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), and tablet computers, as well as fixed terminals such as desktop computers. The controller 600 shown in FIG14 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0177] FIG15 is a structural block diagram of a cleaning device according to an embodiment of the present disclosure.

[0178] As shown in FIG. 15 , a cleaning device 700 includes the controller 600 of the above embodiment.

[0179] In summary, the control method, apparatus, storage medium, controller, and device for cleaning equipment disclosed herein perform image segmentation and edge processing on images of the area to be cleaned captured by a monocular image acquisition element. The segmentation and edge processing results are combined to detect open areas on the ground and key obstacles (such as stains and carpets) in the work scene, construct a target map, and then control the cleaning equipment based on the target map. This improves the efficiency and accuracy of automatic cleaning equipment in complex environments.

[0180] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0181] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0182] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0183] In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation to the present disclosure.

[0184] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0185] In this disclosure, unless otherwise expressly specified or limited, terms such as "installed," "connected," "connect," and "fixed" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integration; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components or interaction between two components, unless otherwise expressly limited. Those skilled in the art will understand the specific meanings of the above terms in this disclosure based on specific circumstances.

[0186] In the present disclosure, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0187] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are illustrative and are not to be construed as limitations on the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A method for controlling a cleaning device, comprising: Obtain data of the area to be cleaned; Processing the data of the area to be cleaned; According to the data processing result, a target map of the area to be cleaned is constructed, wherein the target map includes obstacle information; The cleaning device is controlled according to the target map.

2. The method according to claim 1, wherein: The data of the area to be cleaned includes an image of the area to be cleaned, and processing the data of the area to be cleaned includes: The data of the area to be cleaned are subjected to image segmentation and edge processing, so as to construct a target map of the area to be cleaned according to the edge processing result and the segmentation result.

3. The method according to claim 2, wherein: The image of the area to be cleaned is segmented using a pre-trained segmentation model, and the training process of the segmentation model includes: A plurality of working scene images are acquired; Dividing the plurality of work scene images into a training set and a test set, and respectively marking the targets to be inspected in the work scene images in the training set and the test set, wherein the targets to be inspected include a background, a ground, and traversable obstacles on the ground; A segmentation model is constructed, and the segmentation model is pre-trained using the training set and its corresponding annotation information, and the pre-trained segmentation model is tested using the test set and its corresponding annotation information to obtain a final trained segmentation model.

4. The method according to claim 3, wherein: The step of constructing a target map of the area to be cleaned according to the segmentation results and the edge processing results includes: Performing corrosion processing on the ground area in the segmentation result to obtain a first image; Obtaining an edge line where the background and the ground are in contact in the image of the area to be cleaned according to the edge processing result and the first image; The target map is constructed according to the edge lines and the traversable obstacles on the ground in the segmentation results.

5. The method according to claim 4, wherein: The image of the area to be cleaned is acquired by using an image acquisition element installed on the cleaning device, and the edge line where the background and the ground are in contact in the image of the area to be cleaned is obtained according to the edge processing result and the first image, including: According to the first image, removing the edge in the ground area in the edge processing result; The edge line is determined from the edge processing result after removing the edge in the ground area according to the line of sight of the image acquisition element from near to far.

6. The method according to claim 4, wherein: The step of constructing the target map according to the edge line and the traversable obstacles on the ground in the segmentation result includes: Determine a traversable area according to the edge line, and obtain the type and position of the traversable obstacles in the traversable area according to the traversable area and the traversable obstacles on the ground in the segmentation result; The target map is constructed according to the categories and positions of the traversable obstacles in the traversable area.

7. The method according to claim 1, wherein: The controlling the cleaning device according to the target map includes: Updating the global map according to the target map; Determine a target cleaning area and a target cleaning strategy based on the updated global map; The cleaning device is controlled to clean the target cleaning area according to the target cleaning strategy.

8. The method according to claim 1, wherein: The cleaning device includes a mobile device, and the controlling the cleaning device according to the target map includes: Determining a first obstacle circumvention trajectory of the mobile device; If the first obstacle avoidance trajectory satisfies an update condition, the first obstacle avoidance trajectory is updated to obtain a second obstacle avoidance trajectory; wherein a second obstacle included in the second obstacle avoidance trajectory has different object features from the first obstacle included in the first obstacle avoidance trajectory; The mobile device is controlled to move along the second obstacle avoidance trajectory so that the mobile device avoids the first obstacle and the second obstacle during the movement.

9. The method according to claim 8, wherein: If the first obstacle avoidance trajectory satisfies an update condition, updating the first obstacle avoidance trajectory to obtain a second obstacle avoidance trajectory includes: Controlling the mobile device to move along the first obstacle avoidance trajectory so that the mobile device avoids the first obstacle during the movement; If a second obstacle exists when the mobile device moves along the first obstacle avoidance trajectory, it is determined that the first obstacle avoidance trajectory satisfies the update condition, and the first obstacle avoidance trajectory is updated based on the second obstacle to obtain the second obstacle avoidance trajectory.

10. The method according to claim 8, wherein: The first obstacle includes an obstacle determined based on first sensing data of a first sensor, and / or the second obstacle includes an obstacle determined based on second sensing data of a second sensor.

11. The method according to claim 9, wherein: When controlling the mobile device to move along the first obstacle avoidance trajectory, the method further includes: Obtaining obstacle avoidance data of the mobile device during the movement of the mobile device along the first obstacle avoidance trajectory; If the obstacle avoidance data satisfies the obstacle avoidance condition, it is determined that the obstacle avoidance object corresponding to the obstacle avoidance data is the second obstacle.

12. The method according to claim 11, wherein: The obstacle avoidance data includes posture data, and the obstacle avoidance data satisfies obstacle avoidance conditions, including: Determining a yaw trajectory of the mobile device during movement along the first obstacle avoidance trajectory based on the position and posture data; If the yaw trajectory meets the trajectory characteristics, it is determined that the obstacle avoidance data meets the obstacle avoidance condition.

13. The method according to claim 11, wherein: The obstacle avoidance data includes collision data, and the obstacle avoidance data satisfies obstacle avoidance conditions, including: Determine, based on the collision data, a collision angle and a collision intensity of the mobile device during movement along the first obstacle avoidance trajectory; If the collision angle satisfies a preset angle and the collision intensity satisfies a preset intensity, it is determined that the obstacle avoidance data satisfies an obstacle avoidance condition.

14. The method according to claim 11, wherein: The obstacle avoidance data includes the area and outline corresponding to the obstacle, and the obstacle avoidance data satisfies the obstacle avoidance conditions, including: If the area corresponding to the obstacle satisfies the preset area and the outline satisfies the preset shape, it is determined that the obstacle avoidance data satisfies the obstacle avoidance condition.

15. The method according to claim 14, wherein: When controlling the mobile device to move along the first obstacle avoidance trajectory, the method further includes: Obtaining an area corresponding to an obstacle avoided by the mobile device during movement along the first obstacle avoidance trajectory; If the area is smaller than a preset threshold, it is determined that a second obstacle exists.

16. The method according to claim 8, wherein: The determining of a first obstacle circumvention trajectory of the mobile device includes: Obtaining first obstacle information; An environment map is constructed based on the first obstacle information to generate the first obstacle avoidance trajectory.

17. The method according to claim 8, wherein: If the first obstacle avoidance trajectory satisfies an update condition, updating the first obstacle avoidance trajectory to obtain a second obstacle avoidance trajectory includes: Displaying the first obstacle avoidance trajectory on a human-computer interaction interface; When the mobile device moves along the first obstacle avoidance trajectory, receiving the second obstacle marked for the first obstacle avoidance trajectory on the human-computer interaction interface, and determining that the first obstacle avoidance trajectory satisfies the update condition; The first obstacle avoidance trajectory is updated based on the second obstacle to obtain the second obstacle avoidance trajectory, and the second obstacle avoidance trajectory is displayed on the human-computer interaction interface. The second obstacle avoidance trajectory is shown.

18. The method according to claim 8, wherein: The method further comprises: Obtaining obstacle information when the mobile device moves according to a first map, wherein the first map is the target map; According to the obstacle information, the obstacles on the first map are updated, and the updated obstacles are displayed.

19. The method according to claim 18, wherein: The updated obstacle includes a first obstacle before updating and a newly added second obstacle, and the second obstacle has different object features from the first obstacle.

20. The method according to claim 18, wherein: The updating of obstacles on the first map according to the obstacle information includes: If the obstacle information indicates that a second obstacle exists when the mobile device moves according to the first map, the obstacle on the first map is updated.

21. The method according to claim 19, wherein: The first obstacle includes an obstacle determined based on first sensing data of a first sensor, and / or the second obstacle includes an obstacle determined based on second sensing data of a second sensor.

22. The method according to claim 18, wherein: The obtaining obstacle information when the mobile device moves according to the first map includes: Displaying the first obstacle on the human-computer interaction interface; When the mobile device moves along the first map, a marked second obstacle is received on the human-computer interaction interface to obtain the obstacle information.

23. The method according to any one of claims 19 to 22, wherein: After updating the obstacles on the first map according to the obstacle information, the method includes: The mobile device is controlled to avoid the first obstacle and the second obstacle during movement.

24. A control device for a cleaning device, comprising: An acquisition module is used to acquire data of the area to be cleaned; A processing module, used for processing the data of the area to be cleaned; A construction module, used to construct a target map of the area to be cleaned according to the data processing result, wherein the target map includes obstacle information; A control module is used to control the cleaning device according to the target map.

25. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the control method of the cleaning device according to any one of claims 1 to 23 is implemented.

26. A controller, comprising a memory, a processor and a computer program stored in the memory, wherein when the computer program is executed by the processor, the control method of the cleaning device according to any one of claims 1 to 23 is implemented.

27. A cleaning device, characterized in that: Comprising a controller according to claim 26.

Images