Cleaning device and system

Abstract

A cleaning device includes a position estimation unit configured to estimate a position of the cleaning device, a control unit configured to control the cleaning device so as to autonomously travel and clean a floor surface within a cleaning area based on the position estimated by the position estimation unit, and a cleaning effect determination unit configured to determine a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning. The control unit is configured to, in response to the cleaning effect determination unit determining that dirt on the floor surface has been aggravated, control the cleaning device to cease travel and suspend cleaning in the small area subjected to determination of the cleaning effect.

Description

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application is a continuation application of International Application No. PCT / JP2024 / 017112 filed May 8, 2024 which designated the U.S. and claims priority to Japanese Patent Application No. 2023-081047 filed May 16, 2023, the contents of each of which are incorporated herein by reference.BACKGROUNDTechnical Field

[0002] The present disclosure relates to a cleaning device and a cleaning system.Related Art

[0003] In recent years, cleaning robots that autonomously travel while cleaning floor surfaces have been utilized. These cleaning robots are used for cleaning various areas, including homes, offices, and public facilities (e.g., station yards).BRIEF DESCRIPTION OF THE DRAWINGS

[0004] In the accompanying drawings:

[0005] FIG. 1 is a schematic diagram illustrating an example of a configuration of a cleaning system according to a first embodiment of the present disclosure;

[0006] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a cleaning device according to the first embodiment of the present disclosure;

[0007] FIG. 3 is a functional block diagram illustrating an example of a functional configuration of the cleaning system according to the first embodiment of the present disclosure;

[0008] FIG. 4 is a flowchart illustrating an example of a process flow of a cleaning program according to the first embodiment of the present disclosure;

[0009] FIG. 5 is a flowchart illustrating an example of a process flow of a “cleaning effect determination process” according to the first embodiment;

[0010] FIG. 6 is a flowchart illustrating an example of a process flow of a cleaning program according to a second embodiment of the present disclosure;

[0011] FIG. 7 is a flowchart illustrating an example of a process flow of a “dirt detection process” according to the second embodiment;

[0012] FIG. 8A is a schematic diagram illustrating an example of dirt detection;

[0013] FIG. 8B is a schematic diagram illustrating an example of dirt detection;

[0014] FIG. 9A is a schematic diagram illustrating another example of dirt detection;

[0015] FIG. 9B is a schematic diagram illustrating another example of dirt detection;

[0016] FIG. 9C is a schematic diagram illustrating another example of dirt detection;

[0017] FIG. 10A is a schematic diagram illustrating another example of dirt detection;

[0018] FIG. 10B is a schematic diagram illustrating another example of dirt detection;

[0019] FIG. 11 is a flowchart illustrating an example of a process flow of a “cleaning effect determination process” according to a third embodiment of the present disclosure;

[0020] FIG. 12 is a flowchart illustrating an example of a process flow of a “squash detection process” according to the third embodiment; and

[0021] FIG. 13 is a flowchart illustrating an example of a process flow of a cleaning program according to a fourth embodiment of the present disclosure.DESCRIPTION OF SPECIFIC EMBODIMENTS

[0022] A known cleaning system, as disclosed in WO 2022 / 009391 A1, includes: a first autonomous mobile robot suitable for cleaning a relatively large area; an imaging means for capturing images of the entire cleaning area while the first autonomous mobile robot is cleaning the cleaning area in accordance with cleaning instruction information generated based on layout information of the cleaning area; a generating means for generating uncleaned-area information by analyzing image data generated by the imaging means to determine an uncleaned area within the cleaning area, which has not actually been cleaned by the first autonomous mobile robot; and an instructing means for instructing cleaning of the uncleaned area that is identified from the uncleaned-area information.

[0023] In conventional cleaning systems, such as the cleaning system disclosed in WO 2022 / 009391 A1, cleaning is regarded as completed when the cleaning robot passes through the cleaning area. However, depending on the type of contaminant, such as food, beverage, vomit, or animal excrement, passage of the cleaning robot through the cleaning area may rather spread the dirt instead.

[0024] In view of the foregoing, it is desired to provide a cleaning device and a cleaning system capable of selecting not to perform cleaning when the cleaning device itself would otherwise spread dirt.

[0025] A first aspect of the present disclosure provides a cleaning device including: a position estimation unit configured to estimate a position of the cleaning device; a control unit configured to control the cleaning device so as to autonomously travel and clean a floor surface within a cleaning area based on the position estimated by the position estimation unit; and a cleaning effect determination unit configured to determine a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning. The control unit is configured to, in response to the cleaning effect determination unit determining that dirt on the floor surface has been aggravated, control the cleaning device to cease travel and suspend cleaning in the small area subjected to determination of the cleaning effect.

[0026] A second aspect of the present disclosure provides a cleaning system including: a cleaning device; and a cleaning assistance device that is communicably connected to the cleaning device and is configured to assign to the cleaning device a cleaning task for cleaning a cleaning area. The cleaning device includes: a position estimation unit configured to estimate a position of the cleaning device; a control unit configured to control the cleaning device so as to autonomously travel and clean a floor surface within the cleaning area based on the position estimated by the position estimation unit; and a cleaning effect determination unit configured to determine a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning. The control unit is configured to, in response to the cleaning effect determination unit determining that dirt on the floor surface has been aggravated, control the cleaning device to cease travel and suspend cleaning in the small area subjected to determination of the cleaning effect.

[0027] A third aspect of the present disclosure provides a computer program product including: a non-transitory computer-readable medium; and instructions stored on the non-transitory computer-readable medium that, when executed by at least one processor, causes the at least one processor to implement functions of: estimating a position of a cleaning device; controlling the cleaning device so as to autonomously travel and clean a floor surface within a cleaning area based on the estimated position of the cleaning device; and determining a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning. The controlling the cleaning device includes ceasing travel of the cleaning device and suspending cleaning in the small area subjected to determination of the cleaning effect, in response to determining that dirt on the floor surface has been aggravated.

[0028] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.First EmbodimentCleaning System

[0029] A cleaning system according to the present disclosure will now be described with reference to FIG. 1. As illustrated in FIG. 1, the cleaning system 100 includes a cleaning device 10, which is an autonomous mobile robot, and a cleaning assistance device 12 communicably connected to the cleaning device 10. The cleaning system 100 may include a plurality of cleaning devices 10. In this example, three cleaning devices 10, i.e., cleaning devices 101, 102, and 103, are illustrated. When it is unnecessary to distinguish among the cleaning devices 101, 102, and 103, they are collectively referred to as the cleaning device 10.

[0030] The cleaning assistance device 12 generates, for each of cleaning areas 141, 142, and 143, the content of a cleaning operation (hereinafter referred to as a “cleaning task”) and assigns the cleaning tasks to the cleaning devices 101, 102, and 103. Cleaning tasks for the respective cleaning areas 141, 142, and 143 are assigned to the cleaning devices 101, 102, and 103, respectively. When it is unnecessary to distinguish among the cleaning areas 141, 142, and 143, these areas are collectively referred to as the cleaning area 14. The cleaning device 10 is disposed in the cleaning area 14 to be cleaned and autonomously travels within the area to clean the floor surface thereof.

[0031] In the cleaning system 100 according to the present disclosure, the cleaning device 10 captures images of the floor surface before and after cleaning while autonomously traveling within the cleaning area 14, and determines the cleaning effect by comparing the images of the floor surface before and after the cleaning. When the dirt on the floor surface has been aggravated, it is estimated that the device itself has spread the dirt, and the cleaning device 10 selects not to perform any further cleaning.

[0032] In the present embodiment, an example is described in which the cleaning assistance device 12 generates the tasks. Alternatively, the cleaning device 10 may be equipped with a task generation function. In such a configuration where the cleaning device 10 is equipped with the task generation function, the cleaning assistance device 12 may be omitted.Hardware Configuration

[0033] Next, an example of a hardware configuration of the cleaning device 10 will be described with reference to FIG. 2. As illustrated in FIG. 2, the cleaning device 10 includes a control unit 20. The control unit 20 may be implemented by a computer, and may include a processor 32, a read-only memory (ROM) 34, a random-access memory (RAM) 36, a storage 38, an input / output interface (I / F) 40, and a communication interface (I / F) 42. Each of the processor 32, the ROM 34, the RAM 36, the storage 38, the input / output interface 40, and the communication interface 42 is communicably connected to one another via an internal bus 44.

[0034] The processor 32 may be implemented, for example, as a central processing unit (CPU), and may execute various programs and control respective components. Specifically, the processor 32 reads various programs stored in the ROM 34 or the storage 38, and executes the programs using the RAM 36 as a working area. The processor 32 performs control of respective components constituting the cleaning device 10 and executes various arithmetic operations in accordance with the programs.

[0035] The ROM 34 stores various programs and various data. The RAM 36, as a working area, temporarily stores programs or data. The storage 38 may be implemented by a recording medium such as a hard disk drive (HDD), a solid-state drive (SSD), or a flash memory.

[0036] The storage 38 may store various programs including an operating system, as well as various data necessary for operating the cleaning device 10. In the present embodiment, the ROM 34 or the storage 38 stores a “cleaning program,” which will be described later, and various data necessary for performance of the cleaning tasks.

[0037] The communication interface 42 may be configured to transmit and receive predefined data through wireless communication based on wireless communication standards such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). Through this communication interface 42, the control unit 20 of the cleaning device communicates, via a network, with other electronic devices such as the cleaning assistance device 12 or other cleaning devices 10.

[0038] As illustrated in FIG. 2, a traveling unit 22, a cleaning tool 24, a camera 26, a display 28, and a notification device 30 are connected to the input / output interface 40 of the control unit 20 so that data can be transmitted and received between each of these components and the control unit 20.

[0039] The traveling unit 22 includes a plurality of wheels 22A disposed at front and rear lower portions of the cleaning device 10. At least one of the wheels 22A is rotated by a driving means such as a motor (not illustrated), and the travel direction is variable by a steering means configured to control the direction of at least one of the wheels 22A, whereby autonomous traveling of the cleaning device 10 is implemented.

[0040] The cleaning tool 24 is disposed at an appropriate position of the cleaning device 10 and is a device that enables cleaning of a floor surface while the cleaning device 10 is autonomously traveling. The cleaning tool 24 includes at least a vacuum cleaner capable of suctioning dust and debris on the floor surface. The cleaning tool 24 may further include a fabric or the like capable of performing cleaning such as wet wiping or dry wiping. The cleaning tool 24 may include a robot arm capable of attaching or grasping a predefined cleaning tool. The cleaning tool 24 may incorporate a dust sensor 24A.

[0041] Each cleaning device 10 may include a plurality of cleaning tools 24 of different types. In this case, the cleaning tools 24 may be selectively used according to the types of dirt. Further, each of the plurality of cleaning devices 10 may include the cleaning tool 24 of the same type, or may include the cleaning tool 24 of a different type. In the case where each cleaning device 10 is equipped with the cleaning tool 24 of a different type, for example, cleaning using an optimal cleaning means can be readily performed by changing the cleaning device 10 that performs cleaning according to the type of dirt.

[0042] The camera 26 is an imaging device configured to capture images of the surroundings and the floor surface of the cleaning device 10. The cleaning device 10 estimates its own position based on the images of the surroundings captured by the camera 26. The cleaning device 10 captures images of the floor surface before and after cleaning. For example, the cameras 26 may be installed at the front and rear of the cleaning device 10 to capture images of the floor surface before and after cleaning. For example, the camera 26 installed at the front of the cleaning device 10 captures images of the floor surface ahead of the cleaning device 10 before cleaning (for example, see the illustration of the cleaning device 102 in FIG. 1). After cleaning the floor surface while passing over it, the camera 26 installed at the rear of the cleaning device 10 captures images of the floor surface behind the cleaning device 10 after cleaning (for example, see the illustration of the cleaning device 103 in FIG. 1).

[0043] The camera 26 may be a two-dimensional camera employing an imaging element such as a charge-coupled device (CCD) sensor or a complementary metal-oxide semiconductor (CMOS) sensor. Alternatively, the camera 26 may be a three-dimensional camera such as a stereo camera, a structured-light camera, or a depth-sensor-equipped camera using a time-of-flight (ToF) sensor. Although the camera 26 is illustrated here as an example, an object detection means other than a camera for detecting objects may be used for estimating the position of the cleaning device 10 and for dirt detection described later. For example, instead of or in addition to the camera 26, a radar (e.g., a millimeter-wave sensor) or laser imaging detection and ranging (LiDAR) for detecting the distance and direction to objects around the vehicle, a sonar for detecting objects around the vehicle using sound waves, or a global positioning system (GPS) may be used. Furthermore, the position of the cleaning device 10 may be estimated by utilizing a digital map such as a geographic information system (GIS).

[0044] The display 28 is a device for displaying various items of information to a user or the like. For example, messages or the like to nearby people may be displayed on the display 28. The display 28 may be installed at any position on the cleaning device 10. A known display device such as a liquid crystal display (LCD) or an organic electro-luminescent display (OELD) can be used as the display 28.

[0045] The notification device 30 is configured to notify nearby people of various items of information by emitting sounds or turning on lamps. The notification device may be installed at any position on the cleaning device 10. As the notification device 30, a speaker for outputting sound or a lamp for emitting light may be provided.

[0046] Although detailed description is omitted, the cleaning assistance device 12 is, for example, a server device deployed in the cloud. The cleaning assistance device 12 may be configured as a cloud server capable of providing everything-as-a-service (XaaS). The cleaning assistance device 12 includes a control device, a storage device, and a communication interface. The control device of the cleaning assistance device 12 may be implemented by a computer, similar to the control unit 20 of the cleaning device 10.Functional Configuration

[0047] Next, the functional configuration of the cleaning device 10 will be described with reference to FIG. 3. The cleaning device 10 includes a position estimation unit 50, a dirt detection unit 52, a cleaning effect determination unit 54, a cleanability determination unit 56, a travel control unit 58, a cleaning control unit 60, and a notification control unit 62. The cleaning device 10 of the present disclosure mainly includes, among the illustrated functional blocks, the position estimation unit 50, the cleaning effect determination unit 54, and the cleaning control unit 60 as its principal components. In the following, the functional blocks other than the position estimation unit 50, the dirt detection unit 52, the cleaning effect determination unit 54, and the cleanability determination unit 56 may collectively be referred to as a “control unit.”

[0048] The position estimation unit 50 is a functional block configured to estimate the position of the cleaning device 10. In the present embodiment, the position of the cleaning device 10 is estimated based on the surroundings captured by the camera 26 and map information of the cleaning area 14. The position estimation unit 50 may detect the travel speed of the cleaning device 10 based on changes over time in the position of the cleaning device 10.

[0049] The dirt detection unit 52 is a functional block configured to detect dirt on the floor surface of a portion of the cleaning area 14 before cleaning and determines the type and condition of the dirt. In the present embodiment, the dirt detection unit 52 detects dirt on the floor surface of the cleaning area 14 before cleaning by comparing an image of the floor surface of the cleaning area 14 without dirt, captured beforehand, with an image of the floor surface of the cleaning area 14 captured immediately before cleaning.

[0050] The cleaning effect determination unit 54 is a functional block configured to determine, after cleaning, the cleaning effect on a portion of the cleaning area 14, based on images of the floor surface captured before and after cleaning. In the present embodiment, to determine the cleaning effect, the cleaning effect determination unit 54 compares an image of the floor surface of the cleaning area 14 captured before cleaning with an image captured after cleaning and determines whether the condition of the dirt on the floor surface has been aggravated.

[0051] The cleanability determination unit 56 is a functional block configured to determine, before cleaning, whether the floor surface of a portion of the cleaning area 14 is cleanable. The cleanability determination unit 56 calculates a cleanability degree for the dirt detected by the dirt detection unit 52 and determines, based on the calculated cleanability degree, whether the detected dirt is cleanable by the cleaning device 10.

[0052] The travel control unit 58 is a functional block configured to control the traveling unit 22 based on the position of the cleaning device 10 estimated by the position estimation unit 50, thereby causing the cleaning device 10 to autonomously travel. The cleaning control unit 60 is a functional block configured to perform cleaning of the floor surface within the cleaning area 14 by controlling the position and the like of the cleaning tool 24. In the present embodiment, when the cleaning effect determination unit 54 determines that the condition of the dirt on the floor surface has been aggravated, the cleaning control unit 60 controls the travel control unit 58 to cease travel of the cleaning device 10 and suspends cleaning in the small area where the dirt on the floor surface has determined to have been aggravated.

[0053] In the present embodiment, the operation modes include a “cleaning mode,” in which the cleaning device 10 performs cleaning while autonomously traveling, and a “non-cleaning mode,” in which the cleaning device 10 autonomously travels without performing cleaning. At the beginning of cleaning, the cleaning control unit 60 switches the operation mode from the non-cleaning mode to the cleaning mode. When the cleaning effect determination unit 54 determines that the condition of the dirt on the floor surface has been aggravated, the cleaning control unit 60 switches the operation mode from the cleaning mode to the non-cleaning mode. In the cleaning mode, the travel control unit 58 and the cleaning control unit 60 cooperate to control the cleaning device 10.

[0054] The notification control unit 62 is a functional block configured to cause various items of information to be notified externally, for example, when a cleaning task is suspended. The notification control unit 62 controls the notification device 30 to notify nearby people of various items of information. Further, the notification control unit 62 causes various items of information to be transmitted to the cleaning assistance device 12 and other cleaning devices 10 via the communication interface 42.

[0055] The cleaning assistance device 12 includes a task generation unit 66 and a dirt information storage unit 68. The task generation unit 66 is a functional block configured to generate a task to be performed by each cleaning device 10, specifically, the cleaning task that include details of the cleaning operation. The cleaning task may include information such as the cleaning area to be cleaned, the target degree of cleanliness, and the cleaning tool 24 to be used. The dirt information storage unit 68 is a functional block configured to store, in the storage device, dirt information transmitted from each cleaning device 10 in association with the corresponding position on a map of the cleaning area 14.

[0056] Each functional block of the cleaning device 10 is implemented by execution of a “cleaning program,” which will be described below, by the processor 32 of the control unit 20.Cleaning Program

[0057] Next, the “cleaning program” will be described with reference to FIG. 4. The “cleaning program” is executed by the processor 32 of the control unit 20. The “cleaning program” is initiated upon the cleaning device 10 receiving the cleaning task from the cleaning assistance device 12. Here, the cleaning area 14 is divided into a plurality of small areas, and the cleaning effect is determined for each small area.

[0058] First, at step S100, the processor 32 changes the operation mode of the cleaning device 10 to a cleaning mode for cleaning the floor surface. Next, at step S102, the processor 32 moves the cleaning device 10 to the next small area. At the beginning of cleaning, the processor 32 moves the cleaning device 10 to the small area to be cleaned. The cleaning device 10 acquires surrounding videos from the camera 26, estimates its own position from the surrounding videos, and moves to a desired position.

[0059] Next, at step S104, the processor 32 causes the camera 26 to capture an image of the floor surface of the small area located in front of the cleaning device 10 before cleaning, acquires an image of the floor surface before cleaning, and stores the acquired image in the RAM 36. Next, at step S114, the processor 32 drives the cleaning device 10 to travel and perform cleaning within the small area. Next, at step S116, the processor 32 causes the camera 26 to capture an image of the floor surface of the small area located behind the cleaning device 10 after cleaning, acquires an image of the floor surface after cleaning, and stores the acquired image in the RAM 36. Next, at step S118, the processor 32 performs a “cleaning effect determination process” to determine the cleaning effect in the small area that has been cleaned.

[0060] Here, the “cleaning effect determination process” will now be described with reference to FIG. 5. First, at step S200, the processor 32 acquires an image of the floor surface before cleaning from the RAM 36. Next, at step S202, the processor 32 acquires an image of the floor surface after cleaning from the RAM 36. Subsequently, at step S204, the processor 32 compares the images of the floor surface before and after cleaning. Here, to enable comparison between images captured from any imaging position, the images of the floor surface before and after cleaning are converted into overhead images (Top View), and the two converted images are compared.

[0061] Next, at step S206, the processor 32 determines whether the dirt on the floor surface has spread as a result of the cleaning as compared to before the cleaning. For example, the processor 32 extracts a differential image by subtracting the image of the floor surface before cleaning from the image of the floor surface after cleaning, and when new dirt is added in the differential image, it can be determined that the dirt has spread

[0062] Modes in which the dirt spreads include not only cases where dust or debris is scattered, but also cases where tire marks of the cleaning device 10 remain on the cleaning area 14 during movement. For example, a heavy cleaning device 10 passing over a carpet or flooring may cause significant wear such that the carpet or flooring does not recover even after a certain period of time.

[0063] If the dirt has spread (the answer is YES at step S206), the processor 32 proceeds to step S208. At step S208, the processor 32 makes an aggravation determination, i.e., a determination that the condition of the dirt on the floor surface within the small area has been aggravated, and terminates the routine. On the other hand, if the dirt has not spread (the answer is NO at step S206), the processor 32 terminates the routine without making such an aggravation determination.

[0064] Returning to FIG. 4, at step S120, the processor 32 determines whether the condition of the floor surface within the small area has been aggravated as a result of the cleaning. If it is determined in the above-described “cleaning effect determination process” that the condition of the floor surface of the small area has been aggravated, the floor surface condition is regarded as having been aggravated. If the condition of the floor surface has not been aggravated (the answer is NO at step S120), the processor 32 proceeds to step S122. On the other hand, if the condition of the floor surface has been aggravated (the answer is YES at step S120), the processor 32 proceeds to step S126.

[0065] If the condition of the floor surface has not been aggravated, the processor 32, at step S122, determines whether there is a next small area. If there is a next small area (the answer is YES at step S122), the processor 32 returns to step S102, and repeats the procedures of steps S102 through S122. That is, the processor 32 causes the cleaning device 10 to continue the cleaning operation. If there is no next small area (the answer is NO at step S122), then at step S124, the processor 32 transmits a notification of completion of the cleaning task to the cleaning assistance device 12, and terminates the routine.

[0066] If the condition of the floor surface has been aggravated, the processor 32, at step S126, changes the operation mode of the cleaning device 10 to the non-cleaning mode in which floor cleaning is not performed. That is, the processor 32 causes the cleaning device 10 to suspend the cleaning operation. Subsequently, at step S128, the processor 32 performs a “task suspension process” in a case where the cleaning task has been suspended midway, and thereafter terminates the routine.

[0067] The “task suspension process” may include ceasing the movement of the cleaning device 10 or providing a notification to the surroundings. The method of providing the notification may include emitting a sound or turning on a lamp. The processor 32 may transmit a notification indicating suspension of the cleaning task to the cleaning assistance device 12.

[0068] As described above, in the first embodiment, the cleaning area 14 is divided into a plurality of small areas, and the cleaning effect is determined for each small area. This makes it possible to detect that the condition of the floor surface in any of the plurality of small areas has been aggravated as compared with that before cleaning, even when the cleaning device 10 is actually spreading dirt.

[0069] Further, in the first embodiment, when aggravation of the condition of the floor surface is detected in a small area, the cleaning device 10 is caused to suspend cleaning in that small area, thereby allowing the cleaning to be suspended before the dirt spreads widely within the cleaning area.Second Embodiment

[0070] In a second embodiment, a “dirt detection process” is performed for a small area before cleaning to determine whether the detected dirt is cleanable. When the dirt is determined to be uncleanable, the cleaning operation for that small area is suspended before performing the cleaning. Since the remaining configuration is common to that of the first embodiment, the description of the common portions is omitted, and only the differences will be described.

[0071] Next, with reference to FIG. 6, a “cleaning program” according to the second embodiment will be described. The “cleaning program” according to the second embodiment is identical to the “cleaning program” of the first embodiment in FIG. 4, except that steps S106 through S110 have been added.

[0072] First, at step S100, the processor 32 changes the operation mode of the cleaning device 10 to the cleaning mode for cleaning the floor surface. Next, at step S102, the processor 32 moves the cleaning device 10 to the next small area. Next, at step S104, the processor 32 causes the camera 26 to capture an image of the floor surface of the small area located ahead of the cleaning device 10 before cleaning, acquires the image of the floor surface before cleaning, and stores the acquired image in the RAM 36.

[0073] Next, at step S106, the processor 32 performs a “dirt detection process” to detect dirt on the floor surface before cleaning. Details of the “dirt detection process” will be described later. Next, at step S108, the processor 32 calculates a cleanability degree for the dirt detected at step S106. Next, at step S110, the processor 32 determines whether the cleaning device 10 is capable of cleaning the detected dirt, based on the cleanability degree calculated at step S108. If the detected dirt is cleanable (the answer is YES at step S110), the processor 32 proceeds to step S114. On the other hand, if the detected dirt is uncleanable (the answer is NO at step S110), the processor 32 proceeds to step S126.

[0074] Next, at step S114, the processor 32 drives the cleaning device 10 to travel and perform cleaning within the small area. Next, at step S116, the processor 32 causes the camera 26 to capture an image of the floor surface of the small area located behind the cleaning device 10 after cleaning, acquires the image of the floor surface after cleaning, and stores the acquired image in the RAM 36.

[0075] Next, at step S118, the processor 32 performs a “cleaning effect determination process” to determine the cleaning effect in the cleaned small area. Next, at step S120, the processor 32 determines whether the condition of the floor surface within the small area has been aggravated as a result of the cleaning. If the condition of the floor surface has not been aggravated, the processor 32 proceeds to step S122. On the other hand, if the condition of the floor surface has been aggravated, the processor 32 proceeds to step S126.

[0076] If the condition of the floor surface has not been aggravated, the processor 32, at step S122, determines whether there is a next small area. If there is a next small area, the processor 32 returns to step S102, and the procedures of steps S102 through S122 are repeatedly performed. If there is no next small area, then the processor 32, at step S124, transmits a notification of completion of the cleaning task to the cleaning assistance device 12, and terminates the routine.

[0077] If the condition of the floor surface has been aggravated, or if it has been determined before cleaning that the small area is uncleanable, then at step S126, the processor 32 changes the operation mode of the cleaning device 10 to the non-cleaning mode in which floor cleaning is not performed. Subsequently, at step S128, the processor 32 performs a task suspension process and terminates the routine.

[0078] In the example illustrated in FIG. 6, when it is determined at step S110 before cleaning that the small area is uncleanable, the processor 32 changes the operation mode to the non-cleaning mode. However, as illustrated by the dotted arrow, the uncleanable small area may be skipped, and the cleaning device 10 may move to the next small area. In this case, it is preferable to search for a bypass route that avoids the uncleanable small area and to move the cleaning device 10 to the next small area along the bypass route.Dirt Detection Process

[0079] Here, with reference to FIG. 7, the “dirt detection process” of step S106 in FIG. 6 will be described. First, at step S300, the processor 32 acquires a reference image that has been stored beforehand. The reference image is an image of the corresponding small area captured in a clean state. Next, at step S302, the processor 32 acquires, from the RAM 36, an image of the floor surface before cleaning. Next, at step S304, the processor 32 converts the image into a top-view image and compares the image of the floor surface before cleaning with the reference image.

[0080] Next, at step S306, the processor 32 determines whether dirt is present. The processor 32 also identifies the type of dirt. Here, the type of dirt refers to the type of substance constituting the dirt (hereinafter referred to as “dirt substance”). Examples of dirt substances include dust, paper scraps, sand, mud, footprints, rainwater, food, beverages, vomit, and animal excrement.

[0081] The determination of the type, that is, the class of the dirt substance is performed by an image recognition technique utilizing features of the dirt, such as color, area, shape, and whether it is liquid or solid. For example, if the dirt is three-dimensional, it is determined to be solid, whereas if it is not three-dimensional, it is determined to be liquid. Specifically, when the captured image resembles a reference image in which food is scattered on the floor surface, the dirt substance is determined to be food.

[0082] If dirt is present (the answer is YES at step S306), the processor 32 proceeds to step S308. At step S308, the processor 32 records the position of the dirt. On the other hand, if no dirt is present (the answer is NO at step S306), it is unnecessary to record any dirt position, and the processor 32 skips step S308 and proceeds to step S310. Next, at step S310, the processor 32 outputs the dirt detection result and terminates the routine. When dirt is detected, the dirt detection result includes information such as the presence or absence of dirt, the position of the dirt, and the type of dirt. The dirt detection result may be transmitted to the cleaning assistance device 12, for example, by being included in a notification of cleaning task completion notification or a notification of cleaning task suspension.

[0083] Here, specific examples of dirt detection will be described.Example 1

[0084] FIG. 8 (A) illustrates a reference image 80B. The reference image 80B is an image of a carpet in a clean state. FIG. 8 (B) illustrates an image 80A of the floor surface before cleaning. In the image 80A, a stain 82 formed by a spilled beverage on the carpet is captured. In this case, the stain 82 is detected as a change from the reference image 80B by acquiring a differential image between the reference image 80B and the image 80A of the floor surface before cleaning.Example 2

[0085] FIG. 9 (A) illustrates a reference image 80B1 captured during a first time slot, and FIG. 9 (B) illustrates a reference image 80B2 captured during a second time slot. In the storage 38 of the cleaning device 10, reference images 80B are stored beforehand for respective time slots. FIG. 9 (C) illustrates an image 80A of the floor surface before cleaning, captured during the second time slot. Both the reference image 80B2 captured during the second time slot and the image 80A of the floor surface before cleaning include shadows 84 cast by sunlight or illumination. In this case, in order to prevent the shadows 84 from being erroneously detected as dirt, the reference image 80B2 captured during the same time slot is compared with the image 80A of the floor surface before cleaning. Only the dirt 82 is detected by acquiring a differential image between the two images.Example 3

[0086] FIG. 10 (A) illustrates a reference image 80B, and FIG. 10 (B) illustrates an image 80A of the floor surface before cleaning. In the image 80A of the floor surface before cleaning, a human shadow 86 caused by sunlight or illumination is projected. In the case of a shadow 86 cast by a moving object such as a person or curtain, the boundary between the shadow 86 and the surrounding area moves over time. In this case, to prevent the shadow 86 from being erroneously detected as dirt, the reference image 80B is compared with a plurality of images 80A of the floor surface before cleaning, captured at different times. When the differential image between the two images changes over time, the differential image is determined to represent the shadow 86 and is excluded from detection as dirt.

[0087] Although the above description illustrates examples in which shadows or human silhouettes are identified and excluded during image comparison, the influence of such shadows or silhouettes may also be eliminated by other methods. For example, the images may be captured from a position or angle where shadows or silhouettes are not projected. Alternatively, during imaging, an imaging area may be illuminated by a light source so that shadows or silhouettes are not projected.

[0088] Further, positions of surrounding light sources such as sunlight or illumination may be identified, and the direction and shape of the resulting shadows or silhouettes may be estimated so that the estimated shadows or silhouettes can be removed through image processing. In addition, when an image captured by the camera 26 installed at the rear of the cleaning device 10 includes the shadow of the device itself, the cleaning device 10 may move backward by one area so that the image can be captured by the camera 26 installed at the front of the cleaning device 10.Cleanability Degree

[0089] Here, the calculation of the cleanability degree at step S108 in FIG. 6 and criteria for making a determination as to whether cleaning is possible at step S110 in FIG. 6 will be described. In the second embodiment, a cleanability degree is calculated for the dirt detected in the dirt detection process before cleaning. The cleanability degree is calculated based on the result of the classification determination of the dirt substance.

[0090] In the present embodiment, the cleanability degree is predefined according to the class of the dirt substance. For example, dust and paper scraps are classified as “cleanable,” sand, mud, footprints, and rainwater as “less cleanable,” and food, beverages, vomit, and animal excrement as “uncleanable.” The determination as to whether cleaning is possible is made by threshold evaluation of the calculated cleanability degree—for example, determining that cleaning is possible when the cleanability degree is “cleanable” or “less cleanable.” In such a case, food, beverages, vomit, and animal excrement have a cleanability degree classified as “difficult,” and are therefore determined to be uncleanable. Alternatively, the cleanability degree may also be expressed numerically, such as “cleanable / 80% or higher,”“less cleanable / 50% or higher but less than 80%,” or “uncleanable / less than 50%.”Cleaning Effect Determination Process

[0091] Here, the “cleaning effect determination process” in the second embodiment will be described. The process flow of the “cleaning effect determination process” is the same as that of the “cleaning effect determination process” in the first embodiment shown in FIG. 5. However, when comparing images at step S204 and determining at step S206 whether the dirt has spread, the dirt detection result acquired in the “dirt detection process” at step S106 in FIG. 6 may be taken into account.

[0092] At step S204, the processor 32 acquires the dirt detection result and compares the images of the floor surface before and after cleaning. At step S206, the processor 32 determines, taking into account the dirt detection result, whether the area of the dirt detected before cleaning has expanded as a result of the cleaning. For example, when an area having the same color tone as the detected dirt has spread around the dirt detected in the dirt detection process, it can be determined that the dirt has spread. Further, when the shape of the dirt detected in the dirt detection process has changed, it can be determined that the dirt has spread due to being squashed by the cleaning device.

[0093] As described above, in the second embodiment, the same effects as those in the first embodiment can be achieved.

[0094] Further, in the second embodiment, since dirt is detected before cleaning in a small area, and it is determined whether the detected dirt is cleanable, the cleaning device 10 is caused to suspend cleaning in that small area, thereby preventing the dirt from spreading within the cleaning area beforehand.

[0095] In the second embodiment, since the presence or absence of dirt, the position of the dirt, and the type of dirt are detected before cleaning, the dirt can be reliably cleaned. Moreover, in the dirt detection process, erroneous detection of dirt can be prevented by excluding shadows and silhouettes.

[0096] In the second embodiment, since the dirt detection result is taken into account in the cleaning effect determination, it is possible to determine with higher accuracy whether the dirt has spread.

[0097] In this embodiment, the processor 32 performs the dirt detection process, the calculation of the cleanability degree, and the determination as to whether cleaning is possible before cleaning. Alternatively or additionally, the processor 32 may perform the dirt detection process, calculate the cleanability degree, and determine whether cleaning is possible before proceeding to step S126 due to aggravation of the condition of the floor surface. Further, even when the condition of the floor surface has been aggravated, when it is determined that cleaning is still possible, the processor 32 may cause the cleaning device 10 to clean the same area again.Third Embodiment

[0098] In the second embodiment, the “cleaning effect determination process” is performed taking into account the dirt detection result acquired through the “dirt detection process.” In contrast, in a third embodiment, when the dirt detection result indicates that the shape of the dirt has changed, suggesting the possibility that the dirt has been squashed, a “squash detection process” is performed. Since the remaining configuration is the same as that of the second embodiment, the description of the common portions is omitted, and only the differences will be described.Cleaning Effect Determination Process

[0099] With reference to FIG. 11, the “cleaning effect determination process” in the third embodiment will now be described. First, at step S400, the processor 32 acquires an image of the floor surface before cleaning from the RAM 36. Next, at step S402, the processor 32 acquires an image of the floor surface after cleaning from the RAM 36. Then, at step S404, the processor 32 acquires the dirt detection result and compares the images of the floor surface before and after cleaning.

[0100] At step S406, the processor 32 determines, based on the dirt detection result, whether the shape of the dirt on the floor surface has changed as a result of the cleaning as compared with before the cleaning. If the shape of the dirt has changed (the answer is YES at step S406), the processor 32 proceeds to step S408. On the other hand, if the shape of the dirt has not changed (the answer is NO at step S406), the processor 32 terminates the routine without making the aggravation determination. At step S408, the processor 32 performs a “squash detection process” for detecting that the dirt has been squashed.

[0101] With reference to FIG. 12, the “squash detection process” will now be described. First, at step S500, the processor 32 calculates a friction coefficient μ of the floor surface in the small area being cleaned and stores it in the RAM 36. The friction coefficient of the floor surface can be calculated based on the travel speed of the cleaning device 10. For example, when the cleaning device squashes dirt containing oil, the floor surface becomes slippery, the travel speed increases, and the friction coefficient decreases. In contrast, when the cleaning device squashes sticky dirt, the friction of the floor surface increases, the travel speed decreases, and the friction coefficient increases.

[0102] Next, at step S502, the processor 32 compares the friction coefficient of the floor surface in the small area being cleaned with a previous friction coefficient to determine whether the friction coefficient of the floor surface has changed. The previous friction coefficient may be, for example, the friction coefficient of the immediately preceding small area, or an average value of the coefficients of friction of previously cleaned small areas. If the friction coefficient has changed (the answer is YES at step S502), the processor 32 determines that the dirt has been squashed and proceeds to step S504. On the other hand, if the friction coefficient has not changed (the answer is NO at step S502), the processor 32 determines that the dirt has not been squashed and terminates the routine.

[0103] Next, at step S504, the processor 32 causes the cleaning device 10 to trace back the travel path to a position where the dirt has been squashed. At step S506, the processor 32 causes the camera 26 to capture an image of the floor surface at the position where the dirt has been squashed, acquires the image of the floor surface, and stores the acquired image in the RAM 36. At step S508, the processor 32 performs a classification determination of the dirt substance based on the captured image, calculates the cleanability degree according to the result of the classification determination, and terminates the routine. That is, the type of the squashed dirt is identified, and the cleanability degree is calculated according to the type of dirt.

[0104] Returning to FIG. 11, at step S410, the processor 32 determines whether the dirt has been squashed. If there is a change in the friction coefficient, the processor 32 determines that the dirt has been squashed. If it is not determined that the dirt has been squashed (the answer is NO at step S410), the processor 32 terminates the routine without making the aggravation determination. On the other hand, if it is determined that the dirt has been squashed (the answer is YES at step S410), the process proceeds to step S412.

[0105] Next, at step S412, the processor 32 determines whether the dirt is cleanable by the cleaning device 10, based on the cleanability degree calculated in the “squash detection process.” If the dirt is cleanable (the answer is YES at step S412), the processor 32 terminates the routine without making the aggravation determination. For example, if the squashed substance is paper scraps, it can be determined that cleaning has been performed even though the shape of the dirt has changed from that before cleaning. On the other hand, if the dirt is uncleanable (the answer is NO at step S412), the processor 32 proceeds to step S414. At step S414, the processor 32 makes the aggravation determination and terminates the routine.

[0106] As described above, in the third embodiment, the same effects as those of the first and second embodiments can be achieved.

[0107] Further, in the third embodiment, when the shape of the dirt detected in the “dirt detection process” has changed, it is determined that the dirt has been squashed. When the dirt has been squashed, the cleaning device returns to a position where the dirt has been squashed and identifies the type of the dirt, thereby enabling more appropriate handling of the squashed dirt.

[0108] In the third embodiment, by identifying the type of the dirt that has been squashed and determining again whether the dirt is cleanable, the aggravation determination can be made more prudently. This leads to a reduced number of instances in which cleaning is suspended, thereby improving the cleaning efficiency as compared with a case where the aggravation determination is uniformly made when the dirt is spreading.Fourth Embodiment

[0109] In the first embodiment, when the result of the cleaning effect determination indicates that the condition has not been aggravated, the corresponding small area is regarded as cleaned, and the cleaning device proceeds to a next small area. In contrast, in the fourth embodiment, a detected value from a dust sensor is further acquired, and when dust is detected as being airborne, cleaning of the small area is regarded as incomplete, and the cleaning of the small area is performed again. Since the other configurations are the same as those in the first embodiment, only the differences will be described.

[0110] The “cleaning program” according to the fourth embodiment will be described with reference to FIG. 13. The “cleaning program” according to the fourth embodiment has the same process flow as the “cleaning program” illustrated in FIG. 4, except that steps S130 and S132 are additionally inserted. Accordingly, the same reference numerals are assigned to common portions, and their description will be omitted.

[0111] At step S120, the processor 32 determines whether the condition of the floor surface within the small area has been aggravated due to cleaning. When the condition of the floor surface has not been aggravated (the answer is NO at step S120), the processor 32 proceeds to step S130. On the other hand, when the condition of the floor surface has been aggravated (the answer is YES at step S120), the processor 32 proceeds to step S126.

[0112] If the condition of the floor surface has not been aggravated, the processor 32 acquires a detected value from the dust sensor 24A at step S130. Next, at step S132, the processor 32 determines whether the detected value of the dust sensor 24A is equal to or less than an allowable value. If the detected value of the dust sensor 24A is equal to or less than the allowable value (the answer is YES at step S132), the processor 32 proceeds to step S122. On the other hand, if the detected value of the dust sensor 24A exceeds the allowable value (the answer is NO at step S132), the processor 32 returns to step S114, and the cleaning of the small area is performed again.

[0113] Even if the floor surface becomes clean after cleaning, when dust remains floating in the air, the dust may settle on the floor surface, thereby making it dirty again. In such a case, the processor 32 determines that the cleaning has not been completed and causes the cleaning device 10 to continue cleaning. Alternatively or additionally, the processor 32 may transmit to the cleaning assistance device 12 the detected value of the dust sensor and / or information indicating that cleaning needs to be performed again.

[0114] As described above, in the fourth embodiment, the same effects as those of the first embodiment can be acquired. Furthermore, in the fourth embodiment, since cleaning is continued until the detected value of the dust sensor becomes equal to or less than the allowable value, the floor surface can be substantially cleaned.Fifth Embodiment

[0115] A fifth embodiment relates to a process performed by the cleaning assistance device 12. In the second embodiment, the presence or absence of dirt, the position of the dirt, and the type of dirt are transmitted to the cleaning assistance device 12 as dirt detection results. The dirt detection results may also include captured images of the floor surface or wall surface. The presence or absence of dirt, the position of the dirt, and the type of dirt are plotted on a map, such as a bird's-eye view of the cleaning area 14, and stored in the dirt information storage unit 68 of the cleaning assistance device 12. This allows dirt information to be accumulated for each cleaning area 14.

[0116] With the map stored in the dirt information storage unit 68, it is possible to store information about spots prone to becoming dirty for each cleaning area 14. For example, an area near an entrance or an exit of a room is prone to becoming dirtier than other areas. Accordingly, the cleaning assistance device 12 can generate the next cleaning task while taking into account spots that are prone to becoming dirty. Providing the cleaning device 10 with the dirt information related to the cleaning area 14 together with the cleaning task facilitates cleaning of the cleaning area 14.

[0117] As a result of the accumulation of dirt information, data on age-related stains, wear, and deterioration can be accumulated. The age-related stains, wear, and deterioration include stains that cannot be completely removed, scratches or grooves that remain, fraying of carpet surfaces caused by friction, and sun fading on floor surfaces caused by exposure to sunlight. When dirt information continues to be recorded for the same position in the cleaning area 14 over a certain period, it is presumed that age-related stains, wear, and deterioration have occurred. Accordingly, maintenance information indicating that maintenance such as replacement of carpets or wallpapers is required, that is, maintenance information regarding facilities provided in the cleaning area, may be generated and transmitted to a manager or to another cleaning device 10.Modifications

[0118] The configurations of the cleaning device, cleaning system, and cleaning program described in the above embodiments are merely examples, and it goes without saying that the configurations may be modified without departing from the gist of the present disclosure.

[0119] The control device and the method thereof described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the control device and the method thereof described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control device and the method thereof described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor and memory programmed to perform one or more functions, and a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable, non-transitory tangible storage medium as instructions to be executed by a computer.

Claims

1. A cleaning device comprising:a position estimation unit configured to estimate a position of the cleaning device;a control unit configured to control the cleaning device so as to autonomously travel and clean a floor surface within a cleaning area based on the position estimated by the position estimation unit; anda cleaning effect determination unit configured to determine a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning, whereinthe control unit is configured to, in response to the cleaning effect determination unit determining that dirt on the floor surface has been aggravated, control the cleaning device to cease travel and suspend cleaning in the small area subjected to determination of the cleaning effect.

2. The cleaning device according to claim 1, wherein the cleaning effect determination unit is configured to, when dirt on the floor surface of the small area subjected to determination of the cleaning effect has spread as compared with that before cleaning, determine that dirt on the floor surface has been aggravated.

3. The cleaning device according to claim 1, further comprising a notification device configured to externally notify that cleaning is suspended, whereinthe control unit is configured to, when suspending cleaning, actuate the notification device to externally notify that cleaning is suspended.

4. The cleaning device according to claim 1, further comprising a dirt detection unit configured to detect dirt on a floor surface of a small area to be cleaned before cleaning, whereinthe dirt detection unit is configured to identify presence or absence of dirt, a position of the dirt, and a type of the dirt from an image of the floor surface of the small area to be cleaned.

5. The cleaning device according to claim 4, whereinthe dirt detection unit is configured to detect dirt on the floor surface of the small area to be cleaned by comparing an image of a clean floor surface of the same small area with an image of the floor surface immediately before cleaning.

6. The cleaning device according to claim 4, whereinthe dirt detection unit is configured to detect dirt on the floor surface of the small area to be cleaned by excluding shadows and human silhouettes projected onto the floor surface from the dirt on the floor surface.

7. The cleaning device according to claim 4, further comprising a cleanability determination unit configured to determine whether the floor surface of the small area to be cleaned is cleanable before cleaning, whereinthe cleanability determination unit is configured to calculate a cleanability degree of the small area to be cleaned based on the type of dirt identified by the dirt detection unit, and to determine that the small area to be cleaned is uncleanable when the calculated cleanability degree is equal to or lower than a predefined threshold, andthe control unit is configured to control the cleaning device to suspend cleaning of the small area to be cleaned that has been determined by the cleanability determination unit to be uncleanable.

8. The cleaning device according to claim 7, whereinthe cleanability determination unit is configured to calculate the cleanability degree corresponding to the type of dirt identified by the dirt detection unit, based on a predefined relationship between the type of dirt and the cleanability degree.

9. The cleaning device according to claim 4, whereinthe cleaning effect determination unit is configured to, when the dirt on the floor surface identified by the dirt detection unit has spread in the small area subjected to determination of the cleaning effect as compared with before cleaning, determine that dirt on the floor surface has been aggravated.

10. The cleaning device according to claim 4, whereinthe cleaning effect determination unit is configured to determine that the dirt on the floor surface has been aggravated when squashing of uncleanable dirt is detected.

11. The cleaning device according to claim 10, whereinthe cleaning effect determination unit is configured to, when a shape of the dirt on the floor surface identified by the dirt detection unit in the small area subjected to determination of the cleaning effect differs from that before cleaning, detect that the dirt on the floor surface has been squashed based on a change in a friction coefficient of the floor surface.

12. The cleaning device according to claim 10, whereinthe cleaning effect determination unit is configured to identify a type of squashed dirt from an image of the squashed dirt, and to determine that the squashed dirt is uncleanable when a cleanability degree corresponding to the identified type of the squashed dirt is equal to or lower than a predefined threshold.

13. The cleaning device according to claim 1, further comprising a dust sensor configured to detect dust in air, whereinthe control unit is configured to control the cleaning device to repeatedly perform cleaning of the small area subjected to determination of the cleaning effect until a detected value acquired by the dust sensor becomes equal to or less than a predefined allowable value, regardless of a result of determination by the cleaning effect determination unit.

14. A cleaning system comprising:a cleaning device; anda cleaning assistance device that is communicably connected to the cleaning device and is configured to assign to the cleaning device a cleaning task for cleaning a cleaning area, whereinthe cleaning device comprises:a position estimation unit configured to estimate a position of the cleaning device;a control unit configured to control the cleaning device so as to autonomously travel and clean a floor surface within the cleaning area based on the position estimated by the position estimation unit; anda cleaning effect determination unit configured to determine a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning, whereinthe control unit is configured to, in response to the cleaning effect determination unit determining that dirt on the floor surface has been aggravated, control the cleaning device to cease travel and suspend cleaning in the small area subjected to determination of the cleaning effect.

15. The cleaning system according to claim 14, whereinthe cleaning device further comprises a dirt detection unit configured to detect dirt on the floor surface of the small area to be cleaned before cleaning,the cleaning device is configured to transmit dirt information detected by the dirt detection unit for each small area of the cleaning area to the cleaning assistance device, andthe cleaning assistance device is configured to accumulate, for each small area of the cleaning area, the dirt information received from the cleaning device in a dirt information storage unit.

16. The cleaning system according to claim 15, whereinthe cleaning assistance device is configured to generate at least one of the cleaning task and maintenance information of facilities provided in the cleaning area, based on the dirt information accumulated in the dirt information storage unit.

17. A computer program product comprising:a non-transitory computer-readable medium; andinstructions stored on the non-transitory computer-readable medium that, when executed by at least one processor, causes the at least one processor to implement functions of:estimating a position of a cleaning device;controlling the cleaning device so as to autonomously travel and clean a floor surface within a cleaning area based on the estimated position of the cleaning device; anddetermining a cleaning effect for each small area within the cleaning area based on images of the floor surface before and after cleaning, whereinthe controlling the cleaning device comprises ceasing travel of the cleaning device and suspending cleaning in the small area subjected to determination of the cleaning effect, in response to determining that dirt on the floor surface has been aggravated.

Images