Robot cleaner including light emitter and control method thereof

Abstract

A robot cleaner and a control method thereof are provided. The robot cleaner includes: a camera; a suction unit; a driving unit; and at least one processor acquiring at least one image through the camera while the robot cleaner travels along the first travel route, identifying an amount of dust in one or more regions within a viewing angle of the camera based on the at least one image, and controlling a suction unit to perform a cleaning operation on a cleaning target area among the one or more areas based on the identified dust amount, and acquiring, based on the identified dust amount, first map data indicating a dust amount of a region on which the robot cleaner has traveled in a region within the viewing angle of the camera and second map data indicating a dust amount of a region on which the robot cleaner is to traveled in a region within the viewing angle of the camera.

Description

Technical Field

[0001] This disclosure relates to a robotic cleaner including a light emitter and a control method thereof, and more specifically, to a robotic cleaner and a control method thereof that outputs light toward a bottom surface located in front of the robotic cleaner via the light emitter and identifies the amount of dust present on the bottom surface. Background Technology

[0002] Recently, with the development of electronic technology, robots are being used in various industrial fields. With the development of object recognition technology, robots can correctly distinguish various objects, and with the development of autonomous driving technology, robots can perform stable driving in the driving space without interfering with people passing by.

[0003] Specifically, robots previously used only in the industrial sector are now being used for housework in many homes. Cleaning robots constitute a large proportion of these home robots, and they perform cleaning functions while automatically navigating the home. However, in the case of such robotic cleaners, their mobility is typically limited to a predetermined route, thus they perform cleaning operations on all areas along that route. Consequently, when the robot travels through areas that do not require cleaning (i.e., areas where there is no dust or dirt), it involuntarily performs cleaning operations, resulting in long delays and unnecessary power consumption before the cleaning process is complete. Summary of the Invention

[0004] Technical solution According to one aspect of this disclosure, a robotic cleaner includes: a camera; a vacuum cleaner configured to extract dust from a surface on which the robotic cleaner is placed; a driver configured to move the robotic cleaner; at least one memory storing one or more instructions; and at least one processor configured to execute the one or more instructions, wherein, when executed by the at least one processor, the one or more instructions cause the robotic cleaner to perform the following operations: while the robotic cleaner moves along a first travel path, acquiring at least one image through the camera, identifying the amount of dust in one or more areas within the camera's field of view based on the at least one image, and based on the identified dust... The system determines the amount of dust, controls the vacuum cleaner to perform cleaning operations on the areas to be cleaned in one or more areas, acquires first map data indicating the amount of dust in a first area of ​​the one or more areas, wherein the first area corresponds to the portion of the first travel path that the robot cleaner has previously traveled, acquires second map data indicating the amount of dust in a second area of ​​the one or more areas, wherein the second area corresponds to the portion of the first travel path that the robot cleaner has not yet traveled, and sets a second travel path that may include the area to be cleaned based on the fact that the area to be cleaned is not located on the first travel path, and controls the drive to make the robot cleaner move along the second travel path.

[0005] When executed by the at least one processor, the one or more instructions may also cause the robotic cleaner to perform the following operations: identify areas in the one or more regions where the amount of dust is greater than or equal to a predetermined value as the areas to be cleaned.

[0006] When executed by the at least one processor, the one or more instructions may also cause the robotic cleaner to perform the following operations: acquire second map data before performing the cleaning operation, complete the cleaning operation based on the robotic cleaner following a second travel route, control the drive to return the robotic cleaner to a first travel route, and the first map data may include information on the amount of dust in areas of the one or more regions previously cleaned by the robotic cleaner.

[0007] The robotic cleaner may also include a communication interface, and the one or more instructions, when executed by the at least one processor, may also cause the robotic cleaner to perform the following operations: identify the dust reduction rate of the area to be cleaned based on first map data and second map data, and send the first map data and the identified dust reduction rate of the area to be cleaned to a user terminal device through the communication interface.

[0008] When the one or more instructions are executed by the at least one processor, the robotic cleaner may also perform the following operations: based on the completion of the robotic cleaner's travel along a first travel route; based on first map data, identify at least one area to be re-cleaned in the one or more areas, wherein the at least one area to be re-cleaned has been previously cleaned by the robotic cleaner; set a third travel route including the at least one area to be re-cleaned; and control the drive to move the robotic cleaner along the third travel route.

[0009] When executed by the at least one processor, the one or more instructions may also enable the robotic cleaner to control the vacuum cleaner to vacuum the dust in the area to be cleaned with a suction intensity corresponding to the amount of dust identified in the area to be cleaned.

[0010] When executed by the at least one processor, the one or more instructions may also cause the robotic cleaner to perform the following operations: activating the vacuum cleaner based on the robotic cleaner being located in the area to be cleaned, and controlling the vacuum cleaner to be deactivated based on the robotic cleaner being located in an area other than the area to be cleaned in the one or more areas.

[0011] The robotic cleaner may also include a light emitter configured to emit light, wherein the one or more instructions, when executed by the at least one processor, may also cause the robotic cleaner to perform the following operations: based on the robotic cleaner traveling along a first travel path, emitting light through the light emitter toward a bottom surface located in front of the robotic cleaner, and the at least one image may include at least one image of the bottom surface illuminated by the light.

[0012] When executed by the at least one processor, the one or more instructions may also cause the robotic cleaner to increase the intensity of the light emitted by the light emitter from a first intensity to a second intensity based on the recognition that the area included in the one or more areas within the field of view of the camera is an area previously cleaned by the robotic cleaner.

[0013] According to one aspect of this disclosure, a method for controlling a robotic cleaner includes: acquiring at least one image via a camera of the robotic cleaner while the robotic cleaner is traveling along a first travel path; identifying the amount of dust in one or more areas within the field of view of the camera based on the at least one image; controlling a vacuum cleaner of the robotic cleaner to perform a cleaning operation on an area to be cleaned in the one or more areas based on the identified amount of dust; acquiring first map data indicating the amount of dust in a first area of ​​the one or more areas, wherein the first area corresponds to a portion of the first travel path that the robotic cleaner has previously traveled; and acquiring second map data indicating the amount of dust in a second area of ​​the one or more areas, wherein the second area corresponds to a portion of the first travel path that the robotic cleaner has not yet traveled, wherein controlling the vacuum cleaner includes: identifying the area to be cleaned based on the amount of dust in the area to be cleaned; setting a second travel path that may include the area to be cleaned based on the fact that the area to be cleaned is not located on the first travel path; and controlling a actuator of the robotic cleaner to move the robotic cleaner along the second travel path.

[0014] Controlling the vacuum cleaner may further include: identifying areas in the one or more areas where the amount of dust is greater than or equal to a predetermined value as the areas to be cleaned.

[0015] The method may further include: based on the robot cleaner completing the cleaning operation along the second travel route, controlling the drive to make the robot cleaner return to the first travel route, obtaining second map data may include obtaining second map data before performing the cleaning operation, and obtaining first map data may include: re-identifying the amount of dust in the areas previously cleaned by the robot cleaner in the one or more areas.

[0016] The method may further include: identifying the dust reduction rate of the area to be cleaned based on first map data and second map data; and sending the first map data and the identified dust reduction rate of the area to be cleaned to a user terminal device via a communication interface.

[0017] The method may further include: based on the completion of the robot cleaner's travel along a first travel route; identifying at least one area to be re-cleaned in one or more areas based on first map data, wherein the at least one area to be re-cleaned has been previously cleaned by the robot cleaner; setting a third travel route including the at least one area to be re-cleaned; and controlling the drive to move the robot cleaner along the third travel route.

[0018] According to one aspect of this disclosure, a non-transitory computer-readable storage medium has instructions stored therein that, when executed by at least one processor, cause the at least one processor to perform a method of controlling a robotic cleaner, the method comprising: acquiring at least one image via a camera of the robotic cleaner while the robotic cleaner travels along a first travel path; identifying the amount of dust in one or more areas within the field of view of the camera based on the at least one image; controlling a vacuum cleaner of the robotic cleaner to perform a cleaning operation on an area to be cleaned in the one or more areas based on the identified amount of dust; acquiring first map data indicating the amount of dust in a first area of ​​the one or more areas, wherein the first area corresponds to a portion of the first travel path that the robotic cleaner has traveled; and acquiring second map data indicating the amount of dust in a second area of ​​the one or more areas, wherein the second area corresponds to a portion of the first travel path that the robotic cleaner has not yet traveled, wherein controlling the vacuum cleaner comprises: identifying the area to be cleaned based on the amount of dust in the area to be cleaned; setting a second travel path that may include the area to be cleaned based on the fact that the area to be cleaned is not located on the first travel path; and controlling a actuator of the robotic cleaner to move the robotic cleaner along the second travel path. Attached Figure Description

[0019] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which: Figure 1 This is an example diagram of a robotic cleaner according to an embodiment of the present disclosure; Figure 2 This is a block diagram illustrating the configuration of a robotic cleaner according to an embodiment of the present disclosure; Figure 3 This is a timing diagram schematically illustrating a control method for a robotic cleaner according to an embodiment of the present disclosure; Figure 4 This is an example diagram illustrating a travel route set according to the mode of the robot cleaner in accordance with an embodiment of the present disclosure; Figure 5a and Figure 5b This is an example diagram illustrating a method for generating first and second map data acquired by a robotic cleaner according to an embodiment of the present disclosure; Figure 6 This is a timing diagram schematically illustrating a method for controlling a robotic cleaner to perform cleaning operations on an area not located on a travel path, according to an embodiment of the present disclosure. Figure 7This is an example diagram schematically illustrating a method for controlling a robotic cleaner to perform cleaning operations on an area not located on a travel path, according to an embodiment of the present disclosure; Figure 8 This is an example diagram illustrating the setting of a third driving route for an area to be re-cleaned based on first map data, according to an embodiment of the present disclosure; Figure 9 This is an example diagram illustrating, according to embodiments of the present disclosure, a display on a user terminal of first map data and second map data including dust information of the area to be cleaned acquired by a robotic cleaner; and Figure 10 This is a detailed block diagram of a robotic cleaner according to an embodiment of the present disclosure. Detailed Implementation

[0020] Various modifications can be made to the embodiments described herein, and various types of embodiments may exist. Therefore, specific embodiments are illustrated in the accompanying drawings, and the embodiments will be described in detail in the specific embodiments. However, it should be noted that the various embodiments should not be construed as limiting the scope of this disclosure to the specific embodiments, but rather should be interpreted as including all modifications, equivalents, and / or substitutions of the embodiments of this disclosure. In relation to the detailed description of the drawings, similar components may be indicated by similar reference numerals.

[0021] Furthermore, in describing this disclosure, detailed descriptions will be omitted where it is determined that detailed descriptions of relevant known functions or components may unnecessarily obscure the spirit of this disclosure.

[0022] Furthermore, the embodiments described below can be modified in various different ways, and the scope of the technical concept of this disclosure is not limited to the embodiments described below. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the technical concept of this disclosure to those skilled in the art.

[0023] Furthermore, the terminology used in this disclosure is used to explain specific embodiments of this disclosure and is not intended to limit the scope of other embodiments. Additionally, singular expressions include plural expressions unless clearly defined differently in the context.

[0024] Furthermore, in this disclosure, expressions such as “having,” “may have,” “including,” and “may include” should be interpreted as indicating the presence of such characteristics (e.g., elements such as numerical values, functions, operations, and components), and these terms are not intended to exclude the presence of additional characteristics.

[0025] Furthermore, in this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A and / or B” may include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to all of the following: (1) including A, (2) including B, or (3) including A and B.

[0026] Furthermore, the expressions “first,” “second,” etc., used in this disclosure may be used to describe various elements regardless of any order and / or importance. Moreover, these expressions are used only to distinguish one element from another and are not intended to limit the elements.

[0027] The description of an element (e.g., a first element) in this disclosure being "(operably or communicatively) coupled to another element (e.g., a second element)" / "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element) should be interpreted as including both cases where an element is directly coupled to another element and cases where an element is coupled to another element through yet another element (e.g., a third element).

[0028] Conversely, a description of an element (e.g., the first element) being "directly coupled" or "directly connected" to another element (e.g., the second element) can be interpreted as meaning that there is no other element (e.g., the third element) between the two elements.

[0029] Furthermore, as appropriate, the expression “configured as” as used in this disclosure may be used interchangeably with other expressions such as “suitable for,” “capable of,” “designed for,” “suitable for,” “manufactured as,” and “capable of.” The term “configured as” may not necessarily mean that the device is “specifically designed for” in terms of hardware.

[0030] Conversely, in certain contexts, the phrase "a device configured to..." can refer to a device that is "capable" of performing operations together with another device or component. For example, the phrase "a processor configured to perform A, B, and C" can refer to a dedicated processor (e.g., an embedded processor) for performing the respective operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform the respective operations by executing one or more software programs stored in a memory device.

[0031] Furthermore, in embodiments of this disclosure, a "module" or "component" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. In addition to "modules" or "components" that need to be implemented as specific hardware, multiple "modules" or "components" may be integrated into at least one module and implemented as at least one processor.

[0032] Various elements and areas in the accompanying drawings are shown schematically. Therefore, the technical concept of this disclosure is not limited to the relative dimensions or spacing shown in the drawings.

[0033] In the following, embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings, so that those skilled in the art to which this disclosure pertains can readily implement these embodiments.

[0034] Figure 1 This is an example diagram of a robotic cleaner according to an embodiment of the present disclosure.

[0035] Reference Figure 1 According to an embodiment of the present disclosure, the robotic cleaner 100 emits light toward a bottom surface (e.g., the ground) located in front of the robotic cleaner 100. The robotic cleaner 100 then acquires an image of the bottom surface illuminated by the light and identifies the amount of dust present on the bottom surface based on the acquired image. That is, unlike conventional cleaners (or robotic cleaners) that identify the amount of dust during the dust extraction process, the robotic cleaner 100 according to an embodiment of the present disclosure can identify the amount of dust present on the bottom surface before dust extraction. Therefore, the robotic cleaner 100 according to an embodiment of the present disclosure can perform cleaning operations by selecting only the bottom surface where dust is present.

[0036] Specifically, in the case of traditional robotic cleaners, the robotic cleaner travels only according to a route set for the travel space in which it is located. Therefore, the robotic cleaner performs cleaning operations on all the bottom surfaces along the travel route. As a result, in the case of traditional robotic cleaners, a lot of time is spent completing the route travel, and because cleaning operations are performed on areas where there is no dust, electricity is wasted unnecessarily.

[0037] However, unlike conventional robotic cleaners, the robotic cleaner 100 according to embodiments of this disclosure can pre-identify the presence of dust on its bottom surface. Therefore, the robotic cleaner 100 can selectively perform cleaning operations only on the identified dusty areas. This allows for rapid completion of cleaning operations on the driving space and, specifically, prevents unnecessary power waste.

[0038] In the following text, reference will be made to Figures 2 to 10 This disclosure describes embodiments in this respect.

[0039] Figure 2 This is a block diagram illustrating the configuration of a robotic cleaner according to an embodiment of the present disclosure.

[0040] Reference Figure 2 The robotic cleaner 100 includes a camera 110, a vacuum cleaner 120, a driver 130, and at least one processor 140.

[0041] Camera 110 captures images of objects around the robotic cleaner 100 and acquires multiple images of the objects. Specifically, camera 110 can acquire images of objects (e.g., people, animals, and other objects) present around the robotic cleaner 100.

[0042] Specifically, according to embodiments of this disclosure, camera 110 can capture images of the bottom surface in front of the robotic cleaner 100 and acquire multiple images of the bottom surface. For this purpose, camera 110 can be positioned on the front surface of the robotic cleaner 100 to capture images of the bottom surface in front of the robot.

[0043] Camera 110 can be implemented as an imaging device, such as an imaging device with a CMOS structure (CMOS image sensor (CIS)) or an imaging device with a CCD structure (charge-coupled device). However, this disclosure is not limited thereto, and camera 110 can be implemented as a camera (110) module with various resolutions capable of photographing objects. Furthermore, camera 110 can be implemented as a depth camera, a stereo camera, or an RGB camera, etc. Thus, camera 110 can acquire depth information of an image along with an image of the object.

[0044] Vacuum cleaner 120 is a component that sucks up dust and other particles or substances from a surface where a robotic cleaner is placed, and may include brushes, motors, fans (or rollers), etc. Here, the brush of vacuum cleaner 120 may be made of a material with a low coefficient of friction and good abrasion resistance, such as natural hair or polyamide (PA: nylon), etc.

[0045] In addition, the vacuum cleaner 120 can drive a motor and rotate a fan (or roller) connected to the motor, and when the fan rotates, dust on the surface to be cleaned (e.g., the bottom surface) swept by the brush can be sucked into the interior of the cleaner 100 (e.g., the interior of the body).

[0046] The driver 130 is a component for the mobile robot cleaner 100. The driver 130 may be implemented as wheels, etc. For this purpose, the driver 130 may include a motor. The processor 140 can control the driver 130, thereby controlling various driving operations of the robot cleaner 100, such as moving, pausing, speed control, and direction switching.

[0047] Here, the drive 130 can adjust the driving direction and speed according to the control of the processor 140. For this purpose, the drive 130 may include a power generation device (e.g., depending on the fuel (or energy) used, such as a gasoline engine, diesel engine, liquefied petroleum gas (LPG) engine, electric motor, etc.) for generating power for the movement of the robotic cleaner 100, and a steering device (e.g., manual steering, hydraulic steering, electronic power steering (EPS), etc.) for adjusting the driving direction.

[0048] At least one processor 140 is electrically connected to the camera 110, the vacuum cleaner 120 and the driver 130, and controls the overall operation and function of the robotic cleaner 100.

[0049] At least one processor 140 may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an integrated many-core processor (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. At least one processor 140 may control one or a random combination of other components of the robotic cleaner 100 and perform operations related to communication or data processing. Furthermore, at least one processor 140 may execute one or more programs or instructions stored in memory. For example, at least one processor 140 may perform a method according to embodiments of the present disclosure by executing at least one instruction stored in memory.

[0050] In cases where the method according to embodiments of this disclosure includes multiple operations, the multiple operations may be executed by a single processor or by multiple processors. For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiments, all of the first operation, the second operation, and the third operation may be executed by a first processor, or the first operation and the second operation may be executed by a first processor (e.g., a general-purpose processor), and the third operation may be executed by a second processor (e.g., an artificial intelligence-specific processor).

[0051] At least one processor 140 may be implemented as a single-core processor including one core, or at least one processor 140 may be implemented as one or more multi-core processors including multiple cores (e.g., multiple cores of the same type or multiple cores of different types). When at least one processor 140 is implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include the processor 140's internal memory, such as cache memory, on-chip memory, etc., and a common cache shared by the multiple cores may be included in the multi-core processor. Furthermore, each core (or some of the cores) included in the multi-core processor may independently read and execute program instructions for implementing the methods according to embodiments of the present disclosure, or all of the multiple cores (or some of the cores) may be linked together and read and execute program instructions for implementing the methods according to embodiments of the present disclosure.

[0052] In the case where the method according to embodiments of this disclosure includes multiple operations, the multiple operations may be executed by one of the multiple cores included in a multi-core processor, or the multiple operations may be implemented by multiple cores. For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiments, all of the first operation, the second operation, and the third operation may be executed by the first core included in the multi-core processor, or the first operation and the second operation may be executed by the first core included in the multi-core processor, and the third operation may be executed by the second core included in the multi-core processor.

[0053] In embodiments of this disclosure, processor 140 may refer to a system-on-a-chip (SoC), a single-core processor, a multi-core processor, or a core included in a single-core or multi-core processor, which integrates at least one processor and other electronic components. Furthermore, the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, etc., but embodiments of this disclosure are not limited thereto.

[0054] In the following text, for ease of explanation, at least one processor 140 will be referred to as processor 140.

[0055] According to embodiments of this disclosure, the robotic cleaner 100 may further include a light emitter. Here, the light emitter may be a component that outputs light toward a bottom surface located in front of the robotic cleaner 100. For this purpose, the light emitter may include at least one light-emitting element. For example, the light-emitting element may be a light-emitting diode (LED), a micro LED, etc. Specifically, the light-emitting element may be included in an LED array comprising a plurality of LEDs.

[0056] The light emitter can be positioned lower than the camera 110 positioned on the front surface of the robotic cleaner 100. Therefore, the camera 110 can acquire an image of the bottom surface illuminated by the output light emitted by the light emitter. As an example, the light emitter can be positioned on the robotic cleaner 100 at a height of 0.5 cm to 1 cm above the bottom surface.

[0057] Furthermore, the light emitter may also include lenses, mirrors, etc., to diffuse the light, causing the light emitted from the light emitter to be scattered. Thus, the light emitter can illuminate a wider area of ​​the bottom surface located in front of the robotic cleaner 100. For example, the light emitter can emit light within a 20° range in the vertical direction and a 60° or greater range in the horizontal direction.

[0058] Figure 3 This is a timing diagram schematically illustrating a control method for a robotic cleaner according to an embodiment of the present disclosure.

[0059] Reference Figure 3 During operation S310, while the robot cleaner 100 is traveling along the first travel route, the processor 140 can acquire at least one image via the camera 110. Here, the first travel route can be a travel route set for the travel space where the robot cleaner 100 is located.

[0060] Specifically, the robot cleaner 100 can store predetermined travel route information in its memory. Here, the predetermined travel route can be set based on the size of the travel space where the robot cleaner 100 is located, the position of the robot cleaner 100's station, etc. Furthermore, the travel space can be an indoor space where the robot cleaner 100 is located.

[0061] To set a driving route, the robot cleaner 100 can store map data 10 about the driving space in which it is located in its memory. The processor 140 can acquire environmental information about the driving space (e.g., information about objects within the driving space, such as walls, furniture, etc.) based on images acquired by the robot cleaner 100's camera 110, and generate map data 10 based on the acquired environmental information. Not only can the robot cleaner 100 identify objects (walls, furniture, etc.) within the driving space based on sensing information about the environment of the driving space obtained through sensors (e.g., object distance information, point cloud information about objects, etc.), but it can also generate map data 10 based on information about the identified objects. The processor 140 can generate map data 10 about the driving space using a Simultaneous Localization and Mapping (SLAM) algorithm.

[0062] The processor 140 can set the travel route of the robotic cleaner 100 within its travel space based on the generated map data 10. Here, the processor 140 can set the starting and ending positions of the travel on the map data 10, and then set the optimal travel route for the robotic cleaner 100 to travel from the starting position to the ending position. For example, the processor 140 can set the travel route within the travel space when generating the map data 10 based on a Simultaneous Localization and Mapping (SLAM) algorithm. Alternatively, the processor 140 can set the travel route within the travel space using a local path planning algorithm.

[0063] Figure 4 This is an example diagram illustrating a travel route set according to the mode of the robot cleaner in accordance with an embodiment of the present disclosure.

[0064] The processor 140 can set multiple driving routes according to the mode of the robot cleaner 100. As an example, the mode of the robot cleaner 100 may include a mode in which the robot cleaner 100 performs cleaning while performing driving (hereinafter referred to as the first mode), and a mode in which the robot cleaner 100 performs driving only and detects the area to be cleaned and then selectively cleans only the area to be cleaned (hereinafter referred to as the second mode).

[0065] Here, the size of the vacuum cleaner 120 of the robotic cleaner 100 can be considered when setting the travel route for each of the first and second modes. For example, refer to Figure 4 In the first mode, the interval d1 of the travel path 510 can be set to have a width less than or equal to the width d of the brush 121 of the vacuum cleaner 120. Conversely, in the second mode, the interval d2 of the travel path 520 can be set to have a width greater than the width d of the brush 121 of the vacuum cleaner 120.

[0066] In other words, the interval d1 of the travel route in the first mode can be set to be smaller than the interval d2 of the travel route in the second mode. In the first mode, the robotic cleaner 100 performs cleaning operations while traveling, so a more detailed travel route 510 can be set for the travel space. Conversely, in the second mode, the robotic cleaner 100 quickly detects the area to be cleaned but does not travel while performing cleaning, so the travel route 520 can be set to be less detailed than the travel route 510 in the first mode. Therefore, in the second mode, the robotic cleaner 100 can perform the cleaning operation on the travel space faster than in the first mode.

[0067] The following description assumes that the robotic cleaner 100 travels within a travel space based on a second-mode travel route 520. Therefore, the second-mode travel route 520 will be referred to as the first travel route.

[0068] Reference Figure 4 Map data 10 may include multiple cells (or regions) 11. Here, cell 11 refers to the basic unit of map data 10, which indicates a specific location (or region) in the actual driving space. For example, cells 11 may have the same size and shape as each other (e.g., quadrilateral, triangle, hexagon, polygon, circle, ellipse, etc.) and may be arranged on map data 10 in a grid.

[0069] In other words, each cell 11 on the map data 10 can correspond to each location (or area) in the actual space. Each cell 11 may include information about the probability of an object's possible presence. Therefore, the processor 140 can identify the drivable location or area of ​​the robotic cleaner 100 based on the probability information included in each cell 11, and set a driving route (i.e., a first driving route 510 and a second driving route 520). In this disclosure, for ease of illustration, Figure 5a , Figure 5b , Figure 7 and Figure 8 as well as Figure 4 The probability information (i.e., information about the probability that an object may exist) is not shown in each cell of the map data.

[0070] The processor 140 controls the driver 130 to cause the robotic cleaner 100 to travel along a first travel path 520 stored in memory. Specifically, the processor 140 can rotate the wheels of the driver 130 and thereby perform control to cause the robotic cleaner 100 to travel along the first travel path in the travel space. Then, while the robotic cleaner 100 is traveling, the processor 140 can acquire multiple images of the bottom surface located in front of the robotic cleaner 100 via the camera 110.

[0071] In operation of S320, processor 140 uses the acquired image to determine the amount of dust in the area within the field of view of camera 110.

[0072] Specifically, the processor 140 can identify at least one object in the acquired image. Then, the processor 140 can identify at least one object corresponding to dust among the identified at least one object.

[0073] Here, the processor 140 can identify objects smaller than a predetermined size in the acquired image as dust. For example, the processor 140 can identify points smaller than a predetermined size in the acquired image and identify the identified points as dust on the bottom surface in front of the robot cleaner 100.

[0074] Furthermore, the processor 140 can identify objects in the acquired image that have a size smaller than a predetermined size and a brightness greater than or equal to a predetermined brightness as dust. For example, the processor 140 can identify each pixel value among multiple pixels in the acquired image. Here, the processor 140 can identify pixels with pixel values ​​greater than or equal to a predetermined value as dust. Specifically, the processor 140 can group multiple adjacent pixels with pixel values ​​greater than or equal to the predetermined value, and identify the size of the object corresponding to the multiple pixels based on the number of multiple pixels belonging to the group (or the size of the group including the multiple pixels). Then, if the size of the identified object is smaller than the predetermined size, the processor 140 can identify the object as dust.

[0075] Here, the processor 140 can identify dust based on the difference between a pixel with a pixel value greater than or equal to a predetermined value and the pixel value of another pixel surrounding that pixel. Specifically, the processor 140 can identify the difference in pixel values ​​between another pixel surrounding a pixel with a pixel value greater than or equal to the predetermined value, and if the difference in pixel values ​​is identified to be greater than or equal to the predetermined value, the processor 140 can identify the pixel with a pixel value greater than or equal to the predetermined value as dust.

[0076] Furthermore, the processor 140 can identify dust in an image by inputting the acquired image into a neural network model trained to recognize dust in images and stored in memory. As the output of the neural network model, the processor 140 can obtain the location and amount of dust in the image.

[0077] To this end, processor 140 can train a neural network model based on learning data including multiple images stored in memory. Specifically, the multiple images included in the learning data may include at least one dust particle. Furthermore, each image can be preprocessed such that at least one dust particle included in each image is defined. Processor 140 can train a neural network model to identify dust particles in images based on multiple images.

[0078] For neural network models, various networks can be used, such as deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), restricted Boltzmann machines (RBM), deep belief networks (DBN), deep Q-networks, etc.

[0079] The processor 140 can identify multiple dust particles in an image and then determine the amount of dust present on the bottom surface within the field of view of the camera 110 based on the number of the identified multiple dust particles. For example, the processor 140 can sum the number of multiple dust particles to calculate the amount of dust (or dust concentration) in the area of ​​the bottom surface within the field of view of the camera 110.

[0080] Specifically, the processor 140 can identify multiple units 11 on the map data 10 corresponding to the bottom surface within the field of view of the camera 110, and identify the amount of dust present in each unit. Then, the processor 140 can identify the amount of dust compared to the area of ​​each unit, and identify the dust concentration of each unit.

[0081] According to an embodiment of this disclosure, the processor 140 can use the light emitter of the robot cleaner 100 to illuminate the bottom surface in front of the robot cleaner 100, and acquire an image of the illuminated area through the camera 110, and then identify the amount of dust in the area within the field of view of the camera 110 based on the acquired image.

[0082] Specifically, while the robotic cleaner 100 is moving, the processor 140 can output light towards the front of the robotic cleaner 100 via a light emitter. Then, the processor 140 can acquire an image of the bottom surface illuminated by the output light via a camera 110.

[0083] Specifically, when light emitted by the light emitter illuminates the bottom surface, the processor 140 can more accurately identify dust present on the bottom surface in the image acquired by the camera 110. Here, the processor 140 can identify the pixel value of each pixel in the image and identify dust in the image based on the identified pixel values. Specifically, when light is illuminating, the pixel values ​​of pixels in the image containing dust on the bottom surface may have relatively high values. Therefore, the processor 140 can identify pixels with pixel values ​​greater than or equal to predetermined values ​​as dust. In this respect, the aforementioned embodiments for identifying dust can be applied in the same way, and therefore detailed descriptions will be omitted.

[0084] Then, in operation S330, the processor 140 can identify the amount of dust and control the vacuum cleaner 120 to perform a cleaning operation on the area to be cleaned, wherein the area to be cleaned is identified based on the identified amount of dust.

[0085] Specifically, the processor 140 can identify the area to be cleaned based on the detected amount of dust. Specifically, the processor 140 can identify areas where the detected amount of dust is greater than or equal to a predetermined amount as areas to be cleaned. Here, the area to be cleaned can be identified on the bottom surface within the field of view of the camera 110, which is located in the image.

[0086] To identify areas to be cleaned, processor 140 can group multiple dust particles in an image and identify areas containing multiple groups of dust particles as areas to be cleaned. Specifically, processor 140 can group dust particles whose distances are within a predetermined distance between adjacent dust particles and identify the number of dust particles included in a group. Then, if the number of dust particles included in a group is greater than or equal to a predetermined number, processor 140 can identify that the amount of dust is greater than or equal to the predetermined amount. Then, processor 140 can identify areas to be cleaned on the front bottom surface of the robotic cleaner 100 based on the location of groups in the image that contain more than or equal to the predetermined number of dust particles.

[0087] For example, processor 140 can identify areas in an image containing a predetermined number of dust particles as areas to be cleaned. Then, based on the location of the identified areas to be cleaned in the image, processor 140 can determine the specific location of the areas to be cleaned on the bottom surface in front of the robotic cleaner 100. For example, processor 140 can acquire the coordinate information of the pixels corresponding to the areas to be cleaned in the image and convert the acquired coordinate information into coordinate information of the bottom surface in front of the robotic cleaner 100. Then, processor 140 can determine the location of the areas to be cleaned on the bottom surface in front of the robotic cleaner 100 based on the converted coordinate information. To this end, processor 140 can perform various image processing procedures (such as deformation processing) on ​​the acquired image.

[0088] The processor 140 can identify areas to be cleaned on a unit-by-unit basis. Specifically, the processor 140 can identify multiple units 11 corresponding to the bottom surface within the field of view of the camera 110 in the image map data 10 acquired by the camera 110, and identify the amount or concentration of dust in each unit 11. Then, the processor 140 can identify units 11 among the multiple units 11 whose dust amount is greater than or equal to a predetermined value or whose dust concentration is greater than or equal to a predetermined concentration, and identify the identified units 11 as areas to be cleaned.

[0089] When an area to be cleaned is identified, the processor 140 can control the vacuum cleaner 120 to perform a cleaning operation. Specifically, the processor 140 can control the driver 130 to move the robotic cleaner 100 to the area to be cleaned, and then, when the robotic cleaner 100 is in the area to be cleaned, the processor 140 can control the vacuum cleaner 120 to suck up the dust present in the area to be cleaned.

[0090] Therefore, the robot cleaner 100 of this disclosure can selectively perform cleaning operations only on the areas in the travel space that need to be cleaned (i.e., the areas to be cleaned), thereby enabling the travel space to be cleaned quickly and cleanly.

[0091] Figure 5a and Figure 5b This is an example diagram illustrating a method for generating first map data 20 and second map data 30 acquired by a robotic cleaner 100 according to an embodiment of the present disclosure.

[0092] In operation S340, the processor 140 can acquire map data indicating the amount of dust in the area already traveled by the robot cleaner 100 within the field of view of the camera 110, and map data indicating the amount of dust in the area to be traveled by the robot cleaner 100 within the field of view of the camera 110 based on the identified amount of dust.

[0093] Here, the map data indicating the amount of dust can be map data that includes dust information in the travel space. From the following, for ease of explanation of this disclosure, the map data indicating the amount of dust in the area where the robot cleaner 100 has traveled will be referred to as first map data 20, and the map data indicating the amount of dust in the area where the robot cleaner 100 is about to travel will be referred to as second map data 30.

[0094] The first map data 20 may be map data that includes information about the amount of dust in areas where the robotic cleaner 100 has traveled or performed cleaning operations. Furthermore, the first map data 20 may include information about the amount of dust in areas where dust levels have been identified while the robotic cleaner 100 is traveling. Specifically, even if an area where dust levels have been identified in images acquired by the robotic cleaner 100 is not located on the robotic cleaner 100's travel path, information about the amount of dust in that area may again be included in the first map data 20.

[0095] The second map data 30 may be map data that includes information about the amount of dust in areas where the robotic cleaner 100 has not yet traveled. Specifically, even if areas identified by the images acquired by the robotic cleaner 100 as having high dust levels are not located on the travel route, the second map data 30 may include information about the amount of dust in those areas.

[0096] According to embodiments of this disclosure, processor 140 can identify the amount of dust in areas of the driving space where the robot cleaner 100 is not located based on acquired images. That is, when an image of the bottom surface in front of the robot is acquired by camera 110, and the amount of dust is identified based on the acquired image, processor 140 can identify the amount of dust on the bottom surface in front of the robot cleaner 100.

[0097] Specifically, as the robotic cleaner 100 moves along its travel path, the images acquired by the camera 110 may also include the areas that the robotic cleaner 100 has already traversed. This is because the field of view of the camera 110 may be wider than the intervals of the travel path. Therefore, the processor 140 can re-identify the amount of dust in the areas that the robotic cleaner 100 has traversed.

[0098] The processor 140 can generate first map data 20 and second map data 30 based on map data 10. That is, the processor 140 can identify the amount of dust in map data 10 on a unit basis and generate first map data 20 and second map data 30 on a unit basis. Specifically, the processor 140 can identify the area in front of the robot cleaner 100 based on the robot cleaner 100's position and the acquired image on map data 10, and identify the amount of dust in the identified area on a unit basis in map data 10, generating first map data 20 and second map data 30. Here, the processor 140 can store the first map data 20 and second map data 30 in memory, and update the first map data 20 and second map data 30 whenever the amount of dust in the area within the field of view of camera 110 is identified. Specifically, whenever the amount of dust in a traveled area is identified, the processor 140 can update the first map data 20 stored in memory, and whenever the amount of dust in an untraveled area is identified, the processor 140 can update the second map data 30 stored in memory.

[0099] Furthermore, the processor 140 can generate the first map data 20 by updating (or overlaying) information about the amount of dust identified for the driven areas in the second map data 30. Specifically, information about the amount of dust in undriven areas is included in the second map data 30, and information about the amount of dust in more cells than in the first map data 20 can therefore be included. Thus, if the amount of dust in a driven or previously identified cell 11 is re-identified, the processor 140 can use the information about the re-identified dust amount to update the information about the amount of dust in the cells 11 corresponding to the same location (or area) included in the second map data 30, and can generate the first map data 20.

[0100] Below, we will refer to Figure 5a and Figure 5b The method for generating the first map data 20 and the second map data 30 is described in detail. Each cell 11 included in the map data 10 (or the first map data 20 and the second map data 30) will be referred to as each region. Figure 5a , Figure 5b , Figure 7 and Figure 8 The regions 11' shown in the first map data 20 and 11'' shown in the second map data 30 correspond to the regions 11 included in the same location in the map data 10.

[0101] Reference Figure 5aAt time t1, processor 140 can generate second map data 30 based on images acquired by camera 110. The second map data 30 includes information about the amount of dust in the area where the robotic cleaner 100 will be traversing. Specifically, processor 140 can acquire images of regions A, B, C, D, E, I, J, K, L, and M (specifically, regions corresponding to cells), regions 11-A to 11-E, and regions 11-I to 11-M, as seen from the camera 110's perspective (hereinafter, each region is referred to by its cell number; for example, region A will be referred to as region 11-A, region B as region 11-B, etc.). Then, processor 140 can identify the amount of dust present in regions 11-A to 11-E and regions 11-I to 11-M based on the acquired images. Areas 11-A to 11-E and areas 11-I to 11-M are areas that the robotic cleaner 100 has not yet traversed, and the processor 140 can generate second map data 30 including information about the amount of dust in areas 11-A to 11-E and areas 11-I to 11-M.

[0102] Then, the processor 140 can identify areas to be cleaned in multiple areas located in front of the robotic cleaner 100 based on the generated second map data 30. (See reference...) Figure 5a The processor 140 can identify the amount of dust in multiple areas (areas 11-A to 11-E and areas 11-I to 11-M) and then generate second map data 30 including information about the amount of dust. Here, referring to the second map data 30, the processor 140 can identify the amount of dust present in areas 11-I to 11-K of the multiple areas (areas 11-A to 11-E and areas 11-I to 11-M) as 8, 9, and 2, respectively. Therefore, the processor 140 can identify areas 11-I to 11-K of the multiple areas (areas 11-A to 11-E and areas 11-I to 11-M) as areas to be cleaned.

[0103] Then, the processor 140 can perform cleaning operations only on areas 11-I to 11-K of the multiple areas (areas 11-A to 11-E and areas 11-I to 11-M). Specifically, the processor 140 can control the driver 130 so that the robotic cleaner 100 moves to areas 11-I to 11-K, and when the robotic cleaner 100 moves to areas 11-I to 11-K, the processor 140 can use the vacuum cleaner 120 to suck up the dust.

[0104] Reference Figure 5bThe processor 140 can acquire images of multiple regions (regions 11-A to 11-F and regions 11-I to 11-O) at point t2, located in front of the robotic cleaner 100 (and included in the field of view of the camera 110). The processor 140 can then identify the amount of dust present in the multiple regions (regions 11-A to 11-F and regions 11-I to 11-O) based on the acquired images.

[0105] Here, the processor 140 can identify the amount of dust in the area where the robotic cleaner 100 has traveled based on the acquired images, and generate first map data 20 including information about the amount of dust in the traveled area. The processor 140 can re-identify the amount of dust in the area where the robotic cleaner 100 has traveled based on the acquired images. Specifically, refer to... Figure 5b Regions 11-A to 11-C and regions 11-I to 11-K can be areas that the robot cleaner 100 has already traversed based on its travel route. Here, the traversed area can be the area that overlaps with the robot cleaner 100 when the robot cleaner 100 moves along its travel route.

[0106] Specifically, the areas traversed by the robotic cleaner 100 (areas 11-A to 11-C and areas 11-I to 11-K) may include areas to be cleaned. If areas 11-I to 11-K are included in the image acquired at time point t2, then areas 11-I to 11-K are areas to be cleaned, and are the areas where the robotic cleaner 100 performs cleaning operations immediately following time point t1. Therefore, the processor 140 can re-identify the amount of dust in areas 11-I to 11-K where the robotic cleaner 100 performs cleaning operations based on the image acquired at time point t2.

[0107] The processor 140 can generate first map data 20 based on the amount of dust in areas 11-A to 11-C and areas 11-I to 11-K traveled by the image-recognition robotic cleaner 100 after time point t1, and generate first map data 20 including information about the amount of dust in areas 11-A to 11-C and areas 11-I to 11-K.

[0108] The processor 140 can identify the amount of dust in the area where the robotic cleaner 100 will be traveling based on the acquired images, and generate second map data 30 including information about the amount of dust in the area to be traveled.

[0109] Specifically, refer to Figure 5bRegions 11-E to 11-G and regions 11-M to 11-O may be the areas that the robotic cleaner 100 will travel through after time point t2. The processor 140 may identify the amount of dust in regions 11-E to 11-G and regions 11-M to 11-O based on the acquired images, and generate second map data 30 including information about the identified dust amount. Specifically, the processor 140 may update the previously generated second map data 30 with information about newly identified dust amounts in regions 11-E to 11-G and regions 11-M to 11-O.

[0110] The processor 140 can compare the generated first map data 20 and second map data 30, and identify the results of performing cleaning operations on the areas to be cleaned (areas 11-I to 11-K). Specifically, the processor 140 can identify the rate of reduction of dust, etc., in the areas to be cleaned based on information about the amount of dust in the areas to be cleaned, included in the first map data 20 and the second map data 30.

[0111] Furthermore, the processor 140 can re-identify areas previously identified as areas to be traversed as areas that the robotic cleaner 100 will traverse. That is, the processor 140 can overlap the identification of areas that the robotic cleaner 100 will traverse. Specifically, see... Figure 5a Based on the image acquired at time t1, processor 140 can identify areas 11-D, 11-E, 11-M, and 11-N, which are within the field of view of camera 110, as areas where the robotic cleaner 100 will operate. Therefore, processor 140 can include information about the amount of dust in areas 11-D, 11-E, 11-M, and 11-N in the second map data 30 at time t1. (Refer to...) Figure 5b Based on the image acquired at time t2 after time t1, processor 140 can re-identify regions 11-D, 11-E, 11-M, and 11-N as areas where the robot cleaner 100 will travel. Then, processor 140 can re-identify the amount of dust in regions 11-D, 11-E, 11-M, and 11-N based on the acquired image. Here, if the amount of dust in the area to be traveled changes, processor 140 can update the second map data 30 using information about the most recently acquired dust amount. This is because the amount of dust in the area to be traveled may change as dust is generated according to the environment within the travel space (e.g., the operation of the air conditioner within the travel space).

[0112] Reference Figure 5aAlthough no dust was identified in areas 11-D, 11-E, 11-M, and 11-N of the second map data 30 generated at time t1, dust was identified in areas 11-E and 11-M of the same areas generated at time t2. In other words, the processor 140 can identify the dust levels in areas 11-E and 11-M as 8 and 9 respectively based on the image acquired at time t2, and update the second map data 30 based on the identified dust levels. Therefore, the processor 140 can re-identify areas 11-E and 11-M as areas to be cleaned.

[0113] Figure 6 This is a timing diagram schematically illustrating a method by which a controlled robotic cleaner 100 performs a cleaning operation on an area to be cleaned that is not located on a travel path, according to an embodiment of the present disclosure. Figure 6 Operations S610, S620, and S670 shown herein can be used with Figure 3 Operations S310, S320, and S340 shown in the figure correspond to each other.

[0114] Figure 7 This is an example diagram schematically illustrating a method for controlling a robotic cleaner to perform cleaning operations on an area not located on a travel path, according to an embodiment of the present disclosure.

[0115] According to embodiments of this disclosure, if the area to be cleaned is not located on the first travel route 512, the processor 140 may adjust the first travel route 512. Specifically, the processor 140 may change the first travel route 512 so that the robotic cleaner 100 can move to the area to be cleaned that is not located on the first travel route 512.

[0116] Specifically, the processor 140 can set a new travel route for moving from the position of the robot cleaner 100 to the area to be cleaned at the time point when the area to be cleaned is identified, and adjust the previous first travel route 512 based on the set travel route.

[0117] Reference Figure 6In this regard, the processor 140 can identify areas to be cleaned within the camera's field of view based on the amount of dust obtained from the acquired image during operation S630, and if the area to be cleaned is not located on the travel route, the processor 140 can set a second travel route for the area to be cleaned during operation S640. The second travel route refers to a new travel route from the position of the robot cleaner 100 to the area to be cleaned at the time the area to be cleaned is identified. The second travel route may include a travel route from the position of the robot cleaner 100 at the time the area to be cleaned is identified to the area to be cleaned which is not located on the first travel route 512, and a travel route from the area to be cleaned back to the previous first travel route 512.

[0118] The processor 140 can identify whether the area to be cleaned, identified based on the amount of dust, is located on the travel path. As the robotic cleaner 100 moves along the travel path, the processor 140 can identify whether the area to be cleaned is located on the travel path by determining whether the robotic cleaner 100 can perform cleaning operations on the area to be cleaned.

[0119] Specifically, the processor 140 can consider not only the first travel path 512 set for the robotic cleaner 100, but also the size, width, etc. of the vacuum cleaner 120 to identify whether the area to be cleaned is located on the first travel path 512. For example, if the processor 140 identifies that the vacuum cleaner 120 of the robotic cleaner 100 can be located on the area to be cleaned while the robotic cleaner 100 is moving along the first travel path 512, then the processor 140 can identify that the area to be cleaned is located on the first travel path 512.

[0120] Conversely, if the robot cleaner 100 continues to move along the first travel path 512, and if it is determined that the vacuum cleaner 120 of the robot cleaner 100 is not located on the area to be cleaned, the processor 140 can determine that the area to be cleaned is not located on the first travel path 512.

[0121] If it is determined that the area to be cleaned is not located on the first travel route 512, the processor 140 may set a new travel route from the position of the robot cleaner 100 at the time the area to be cleaned was identified (e.g., the position of the robot cleaner 100 on the first travel route 512 at the time the area to be cleaned was identified) to the area to be cleaned.

[0122] For example, refer to Figure 5b and Figure 7The processor 140 can newly identify areas 11-E and 11-M as areas to be cleaned based on the updated second map data 30. Here, the processor 140 can identify that the locations of the newly identified areas 11-E and 11-M are not located on the first travel route 512, based on the positions of the newly identified areas 11-E and 11-M and the predetermined first travel route 512. Therefore, the processor 140 can set a second travel route 513, through which the robotic cleaner 100 can travel to areas 11-E and 11-M. Here, the second travel route 513 can be the travel route taken by the robotic cleaner 100 from the previous first travel route 512 to areas 11-E and 11-M. Then, the processor 140 can adjust the first travel route 512 based on the set second travel route 513. Specifically, the processor 140 can adjust the previous first travel route 512 to include the set second travel route 513.

[0123] Then, in operation S650, the processor 140 can control the driver 130 to cause the robot cleaner 100 to deviate from the first travel route 512 and travel along the set second travel route 513 to the area to be cleaned.

[0124] The processor 140 can control the drive 130 to cause the robotic cleaner 100 to travel along a set second travel route 513 to the area to be cleaned. Then, when the cleaning operation of the area to be cleaned is completed, the processor 140 can control the drive 130 to cause the robotic cleaner 100 to return to the previous first travel route 512.

[0125] Here, before cleaning an area not located on the first travel route 512, the processor 140 can acquire second map data 30 based on the amount of dust in the area to be cleaned according to the acquired images. Then, when the cleaning operation on the area to be cleaned is completed along the second travel route 513, the processor 140 can control the driver 130 to cause the robot cleaner 100 to return to the first travel route 512 and then travel along the first travel route 512. Then, while the robot cleaner 100 is traveling along the first travel route 512, if the amount of dust in the area to be cleaned is re-identified based on at least one image acquired by the camera 110, the processor 140 can acquire first map data 20 based on the re-identified amount of dust. In this regard, regarding Figure 5a and Figure 5b The foregoing description of this disclosure is applied in the same way, therefore detailed description will be omitted.

[0126] According to an embodiment of this disclosure, when the robot completes its journey along the first travel route 512, the processor 140 can identify at least one area to be re-cleaned among a plurality of areas to be cleaned based on the identified amount of dust. The processor 140 can then set a third travel route 514 for the identified at least one area to be re-cleaned and control the driver 130 to cause the robot cleaner 100 to travel to the at least one area to be re-cleaned according to the set third travel route 514.

[0127] Specifically, when the robot cleaner 100 completes its journey based on the first travel route 512 (or an adjusted first travel route), the processor 140 can identify the re-identified dust levels in multiple areas to be cleaned based on the first map data 20. Then, the processor 140 can identify areas among the multiple areas to be cleaned where the robot cleaner 100 will perform cleaning operations again, based on the re-identified dust levels. Specifically, the processor 140 can identify areas to be cleaned where the re-identified dust levels are greater than or equal to a predetermined value or where the dust reduction rate is less than a predetermined value, based on the first map data 20 and the second map data 30, as areas to be cleaned again.

[0128] Then, the processor 140 can set a third travel route 514 for the area to be re-cleaned, so that the robotic cleaner 100 can perform cleaning operations only on the identified areas to be re-cleaned. Specifically, since the third travel route 514 is set only for the selected areas to be re-cleaned among the multiple areas to be cleaned, the shape and length of the third travel route 514 can be simplified or shorter than the first travel route 512.

[0129] Then, the processor 140 can perform a re-cleaning operation on the area to be re-cleaned based on the set third travel route 514. Specifically, the processor 140 can control the driver 130 so that the robot cleaner 100 travels along the third travel route 514 to at least one area to be re-cleaned, and when the robot cleaner 100 moves to the area to be re-cleaned, the processor 140 can use the vacuum cleaner 120 to suck up the dust.

[0130] Figure 8 This is an example diagram illustrating the setting of a third driving route 314 for an area to be re-cleaned based on first map data 20, according to an embodiment of the present disclosure.

[0131] Reference Figure 8The processor 140 can identify six areas (areas 11-J, 11-M, 11-E, 11-Q, 11-R, 11-S, 11-T, 11-U, 11-V, 11-W, 11-X, 11-Y, and 11-Z) from the fifteen areas to be cleaned (areas 11-I, 11-J, 11-K, 11-M, 11-E, 11-Q, 11-R, 11-S, 11-T, 11-U, 11-V, 11-W, 11-X, 11-Y, and 11-Z) that still contain dust as areas to be cleaned again. Then, the processor 140 can set a third travel route 514 for the six areas to be re-cleaned (area 11-J, area 11-M, area 11-E, area 11-S, area 11-U, and area 11-Y), and perform cleaning operations on the areas to be re-cleaned (area 11-J, area 11-M, area 11-E, area 11-S, area 11-U, and area 11-Y) based on the set third travel route 514.

[0132] Specifically, the areas to be re-cleaned by the robotic cleaner 100 may include not only areas identified based on the second map data 30, but also new areas not previously identified as areas to be cleaned. Specifically, depending on the environment within the travel space where the robotic cleaner 100 is located, dust may be newly present in areas not previously identified as areas to be cleaned by the robotic cleaner 100. Therefore, the processor 140 can re-identify the amount of dust in areas already traveled by the robotic cleaner 100 using the first map data 20, which includes information about the amount of dust in areas previously traveled by the robotic cleaner 100, and identify areas to be re-cleaned not only in previously identified areas but also in areas not previously identified as areas to be cleaned.

[0133] The processor 140 can control the vacuum cleaner 120 to suction dust from the area to be cleaned at a suction intensity corresponding to the amount of dust in the area. In other words, the processor 140 can set the suction intensity of the vacuum cleaner 120 in the area to be cleaned to be proportional to the amount of dust. For example, when the amount of dust is greater, the processor 140 can set the suction intensity of the vacuum cleaner 120 to be stronger. Therefore, when multiple areas to be cleaned are identified in the driving space, the suction intensity of the vacuum cleaner 120 can be set to be proportional to the amount of dust present in each area.

[0134] In the case of a conventional robotic cleaner 100, the cleaning operation is performed with a pre-set suction intensity. Specifically, in the case of a conventional robotic cleaner 100, the amount of dust is not pre-identified by image, but rather identified during the dust suction process. Therefore, it is impossible to set the suction intensity to correspond to the pre-identified amount of dust. However, the robotic cleaner 100 according to this disclosure can pre-identify the amount of dust on the bottom surface (especially the area to be cleaned) located in front of the robot, and therefore a suction intensity corresponding to the amount of dust can be set. As a result, the battery power consumption of the robotic cleaner 100 can be saved.

[0135] The processor 140 can turn off the vacuum cleaner 120 while the robot cleaner 100 is moving (i.e., keep the vacuum cleaner in a disabled state), and turn on the vacuum cleaner 120 when the robot cleaner 100 is in an area to be cleaned (i.e., activate the vacuum cleaner). In other words, when the robot cleaner 100 is moving along its path, the processor 140 can prevent the vacuum cleaner 120 from operating by turning it off. Then, the processor 140 will only perform the cleaning operation by turning on the vacuum cleaner 120 when it detects that the robot cleaner 100 has moved to and is in an area to be cleaned. This saves battery power for the robot cleaner 100.

[0136] According to an embodiment of this disclosure, if the area to be cleaned after the cleaning operation is identified to be included within the field of view of the camera 110, the processor 140 may increase the intensity of the light output by the light emitter from a first intensity to a second intensity.

[0137] Specifically, the processor 140 can identify whether the area to be cleaned after the cleaning operation is completed is included in the field of view of the camera 110 based on the image acquired by the camera 110. Then, if it is identified that the area to be cleaned after the cleaning operation is completed is included in the field of view, the processor 140 can increase the intensity of the light output through the light emitter. Specifically, the processor 140 can increase the intensity of the light output through the light emitter from a first intensity to a second intensity. To do this, the processor 140 can increase the driving current sent to the light-emitting element included in the light emitter.

[0138] Figure 9 This is an example diagram illustrating, according to an embodiment of the present disclosure, a display on a user terminal of first map data 20 and second map data 30 including dust information of the area to be cleaned acquired by the robotic cleaner 100.

[0139] According to embodiments of this disclosure, whenever the first map data 20 and the second map data 30 are updated, the processor 140 can send the first map data 20 and the second map data 30 to the user terminal 200 via a communication interface. Here, the processor 140 can display a GUI, image objects, etc., so that the amount of dust identified on the area to be cleaned, included in the first map data 20 and the second map data 30, is displayed, or the user can identify the area to be cleaned, and the amount of dust identified on the area to be cleaned, included in the first map data 20 and the second map data 30, is sent to the user terminal 200. For example, the processor 140 can display a color only for the area to be cleaned among multiple areas included in the second map data 30, so that the user can intuitively identify the area to be cleaned. Here, the processor 140 can display different colors for the area to be cleaned according to the amount of dust. For example, in the area to be cleaned, if the amount of dust is greater than or equal to a first value and less than a second value, the color can be displayed as yellow; if the amount of dust is greater than or equal to the second value and less than a third value, the color can be displayed as red; and if the amount of dust is greater than or equal to the third value, the color can be displayed as black.

[0140] The processor 140 can identify the dust reduction rate of the area to be cleaned based on information about the amount of dust in the area to be cleaned, included in the first map data 20 and the second map data 30. The processor displays the identified dust reduction rate of the area to be cleaned in the first map data 20 and sends the identified dust reduction rate of the area to be cleaned to the user terminal 200 via a communication interface. Thus, the user can identify the cleaning efficiency of the robotic cleaner 100 in the area to be cleaned.

[0141] Reference Figure 9 Whenever the first map data 20 and the second map data 30 are updated, the user terminal 200 can receive the first map data 20 and the second map data 30 from the robot cleaner 100 in real time. Specifically, the first map data 20, which includes information about the amount of dust in the areas that have been traversed, can be displayed on the user terminal 200 in the form of a cleaning result report for the areas to be cleaned, and the second map data 30, which includes information about the amount of dust in the areas that the robot cleaner 100 will traverse, can be displayed on the user terminal 200 in the form of an identification report for the areas to be cleaned.

[0142] The processor 140 can periodically calculate the number of times each area is identified as an area to be cleaned based on the second map data 30. Then, for areas that are identified as areas to be cleaned more than or equal to a predetermined number of times, the processor 140 can designate the area as an area requiring attention and send information about the area requiring attention to the user terminal through a communication interface. Specifically, the processor 140 can display the location of the area requiring attention on map data corresponding to the driving space and send the map data and a message requesting the user to take action on the area requiring attention to the user terminal through a communication interface.

[0143] Figure 10 This is a detailed block diagram of a robotic cleaner 100 according to an embodiment of the present disclosure.

[0144] Reference Figure 10 The robotic cleaner 100 includes a camera 110, a vacuum cleaner 120, a driver 130, a light emitter 150, a display 160, a sensor 170, a user interface 180, a communication interface 190, and at least one processor 140. (Regarding...) Figure 2 For components that are repeated, detailed descriptions will be omitted.

[0145] The light emitter 150 may be a component that outputs light toward a bottom surface located in front of the robotic cleaner 100. For this purpose, the light emitter 150 may include at least one light-emitting element. For example, the light-emitting element may be a light-emitting diode (LED), a micro LED, etc. Here, the light-emitting element may include an LED array, wherein the LED array includes multiple LEDs.

[0146] The display 160 displays various image information under the control of the processor 140. Here, the images can be of various formats, such as text, still images, moving images, graphical user interfaces (GUIs), etc. Specifically, the display 160 can display first map data and second map data.

[0147] Furthermore, the display 160 can be implemented as a touchscreen together with the touch panel. Here, the display 160 can be used as an output device for outputting information between the robot cleaner 100 and the user, and at the same time as an input device for providing an input interface between the robot cleaner 100 and the user.

[0148] Therefore, the robotic cleaner 100 may include various types of displays 160 capable of displaying images, such as liquid crystal displays (LCDs), light-emitting diodes (LEDs), and plasma display panels (PDPs). Depending on the implementation method, the display 160 may additionally include supplementary components. For example, if the display 160 employs a liquid crystal method, the display 160 may include an LCD display panel, a backlight unit that provides light to the display 160, and a panel driver board that drives the panel.

[0149] Sensor 170 can acquire various sensing information around the robotic cleaner 100. As an example, sensor 170 can acquire distance information of objects (e.g., walls, furniture, electronic devices, etc.) located around the robotic cleaner 100. Based on the sensing information acquired by sensor 170, processor 140 can generate map data for the driving space where the robotic cleaner 100 is located. Then, based on the generated map data, processor 140 can generate first map data 20 and second map data 30, including information about the amount of dust. For this purpose, sensor 170 can be implemented as a ToF sensor, LiDAR sensor, infrared sensor, etc.

[0150] User interface 180 is a component for the robotic cleaner 100 to interact with a user, and processor 140 can receive various information inputs through user interface 180. User interface 180 may include at least one of touch sensor 170, motion sensor 170, button, micro switch, switch, microphone, or speaker, but this disclosure is not limited thereto.

[0151] The communication interface 190 can communicate with external devices (e.g., user terminal 200) and external servers through various communication methods. The communication connection between the communication interface 190 and the external devices and servers may include communication via a third device (e.g., a repeater, hub, access point, gateway, etc.). For example, the external device may be implemented as another electronic device, a server, cloud storage, a network, etc.

[0152] Communication interface 190 may include various communication modules for performing communication with external devices. As an example, communication interface 190 may include a wireless communication module, and may include a cellular communication module using at least one of, for example, third-generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), LTE-A Advanced, Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunications System (UMTS), WiBro, or Global System for Mobile Communications (GSM). As another example, the wireless communication module may include at least one of, for example, Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), or Zigbee.

[0153] In addition to the above, the robotic cleaner 100 may also include a memory. The memory may store data required for various embodiments of this disclosure. Specifically, the memory may store map data 10 and travel routes (e.g., information about a first travel route 512) of the robotic cleaner 100 according to embodiments of this disclosure.

[0154] Depending on the use of the stored data, the memory can be implemented as a memory embedded in the robot cleaner 100, or as a memory that can be attached to or detached from the robot cleaner 100. For example, in the case of data used to operate the robot cleaner 100, the data can be stored in a memory embedded in the robot cleaner 100, and in the case of data used for extended functions of the robot cleaner 100, the data can be stored in a memory that can be attached to or detached from the robot cleaner 100.

[0155] Regarding the memory embedded in the robotic cleaner 100, the memory may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)).

[0156] Furthermore, in the case of a memory that can be attached to or removed from the robot cleaner 100, the memory can be implemented in the form of a memory card (e.g., a compact flash memory (CF), a secure digital card (SD), a micro-secure digital card (Micro-SD), a mini secure digital card (Mini-SD), an extreme digital card (xD), a multimedia card (MMC), etc.) and an external memory that can be connected to a USB port (e.g., a USB memory 130).

[0157] The methods according to the various embodiments of this disclosure can be implemented as an application that can be installed on a conventional robotic cleaner 100. Alternatively, the methods according to the various embodiments of this disclosure can be performed using a trained neural network (or a deeply trained neural network) based on deep learning (i.e., a learned network model). Furthermore, the methods according to the various embodiments of this disclosure can be implemented simply by upgrading the software or hardware of the conventional robotic cleaner 100. Additionally, the various embodiments of this disclosure can also be performed via an embedded server disposed on the robotic cleaner 100 or an external server of the robotic cleaner 100.

[0158] According to embodiments of this disclosure, the foregoing various embodiments can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). A machine refers to an apparatus that invokes and operates according to the instructions stored in the storage medium, and the apparatus may include a display device (e.g., display device A) according to embodiments of this disclosure. Where the instructions are executed by a processor, the processor may perform the function corresponding to the instructions itself, or, under the control of the processor, perform the function corresponding to the instructions by using other components. Instructions may include code generated by a compiler or code executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" means only that the storage medium does not include signals and is tangible, but does not indicate whether data is stored semi-permanently or temporarily in the storage medium.

[0159] Furthermore, according to embodiments, the methods according to the foregoing various embodiments may be provided when included in a computer program product. A computer program product refers to a product that can be traded between a seller and a buyer. The computer program product may be distributed online in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or through an app store (e.g., the Play Store). TM The computer program product may be distributed online. In the case of online distribution, at least a portion of the computer program product may be stored at least temporarily in a storage medium (such as the manufacturer's server, the app store's server, and the storage of a forwarding server), or at least a portion of the computer program product may be temporarily generated.

[0160] Furthermore, each component (e.g., module or program) according to the various embodiments described above may consist of a single object or multiple objects. Additionally, some sub-components may be omitted in the corresponding sub-components described above, or other sub-components may be included in the various embodiments. Optionally or additionally, some components (e.g., module or program) may be integrated into an object and perform the functions performed by each component prior to integration in the same or similar manner. Modules, programs, or operations performed by other components according to the various embodiments may be performed sequentially, in parallel, repeatedly, or heuristically. Alternatively, at least some operations may be performed in a different order or omitted, or other operations may be added.

[0161] Furthermore, while embodiments of this disclosure have been shown and described, this disclosure is not limited to the specific embodiments described above, and it will be apparent to those skilled in the art that various modifications may be made without departing from the spirit of this disclosure as claimed in the appended claims. Moreover, such modifications are intended to be interpreted without independence from the technical concept or vision of this disclosure.

Claims

1. A robotic cleaner, comprising: camera; A vacuum cleaner is configured to suck up dust from the surface on which the robotic cleaner is placed; A driver is configured to move the robotic cleaner. At least one memory, storing one or more instructions; as well as At least one processor is configured to execute the one or more instructions. Wherein, the one or more instructions, when executed by the at least one processor, cause the robotic cleaner to perform the following operations: As the robotic cleaner moves along the first travel path, it acquires at least one image via the camera. Based on the at least one image, the amount of dust in one or more areas within the camera's field of view is identified. Based on the identified amount of dust, the vacuum cleaner is controlled to perform cleaning operations on the areas to be cleaned in one or more of the designated areas. Acquire first map data indicating the amount of dust in a first area among the one or more areas, wherein the first area corresponds to a portion of the first travel route previously traversed by the robotic cleaner. Acquire second map data indicating the amount of dust in a second area among the one or more areas, wherein the second area corresponds to the portion of the first travel route that the robotic cleaner has not yet traversed, and Since the area to be cleaned is not located on the first travel route, a second travel route including the area to be cleaned is set, and the drive is controlled to make the robot cleaner move along the second travel route.

2. The robotic cleaner as described in claim 1, wherein, When executed by the at least one processor, the one or more instructions also cause the robotic cleaner to perform the following operations: The area in which the amount of dust in one or more of the regions is greater than or equal to a predetermined value is identified as the area to be cleaned.

3. The robotic cleaner as described in claim 1, wherein, When executed by the at least one processor, the one or more instructions also cause the robotic cleaner to perform the following operations: Acquire second map data before performing the cleaning operation. Based on the fact that the robotic cleaner completes the cleaning operation along the second travel path, the actuator is controlled to return the robotic cleaner to the first travel path, and The first map data includes information on the amount of dust in areas previously cleaned by the robotic cleaner within the one or more areas.

4. The robotic cleaner as described in claim 3, further comprising: Communication interface, Wherein, the one or more instructions, when executed by the at least one processor, also cause the robotic cleaner to perform the following operations: The dust reduction rate of the area to be cleaned is identified based on the first map data and the second map data. The first map data and the dust reduction rate of the identified area to be cleaned are sent to the user terminal device through the communication interface.

5. The robotic cleaner as described in claim 3, wherein, When executed by the at least one processor, the one or more instructions also cause the robotic cleaner to perform the following operations: Based on the completion of the robot cleaner's journey along the first travel route: Based on the first map data, at least one area to be re-cleaned is identified in one or more of the regions, wherein the at least one area to be re-cleaned has been previously cleaned by the robotic cleaner. The setup includes a third driving route for at least one area to be re-cleaned, and The actuator is controlled to move the robotic cleaner along a third travel path.

6. The robotic cleaner as claimed in claim 1, wherein, When executed by the at least one processor, the one or more instructions also cause the robotic cleaner to perform the following operations: The vacuum cleaner is controlled to suction dust from the area to be cleaned at a suction intensity corresponding to the amount of dust identified in the area to be cleaned.

7. The robotic cleaner as described in claim 2, wherein, When executed by the at least one processor, the one or more instructions also cause the robotic cleaner to perform the following operations: Based on the robot cleaner being located in the area to be cleaned, the vacuum cleaner is activated, and Based on the fact that the robotic cleaner is located in an area other than the area to be cleaned in one or more of the areas, the vacuum cleaner is controlled to be in a deactivated state.

8. The robotic cleaner of claim 1, further comprising: A light emitter is configured to emit light. Wherein, the one or more instructions, when executed by the at least one processor, also cause the robotic cleaner to perform the following operations: As the robotic cleaner travels along a first travel path, it emits light through the light emitter toward the bottom surface located in front of the robotic cleaner. The at least one image includes at least one image of the bottom surface illuminated by the light.

9. The robotic cleaner as claimed in claim 8, wherein, When the one or more instructions are executed by the at least one processor, the robotic cleaner also performs the following operations: Based on the identification that the area included in one or more regions within the camera's field of view is an area previously cleaned by the robotic cleaner, the intensity of the light emitted by the light emitter is increased from a first intensity to a second intensity.

10. A method for controlling a robotic cleaner, the method comprising: As the robotic cleaner travels along the first travel route, at least one image is acquired through the camera of the robotic cleaner. Based on the at least one image, identify the amount of dust in one or more areas within the camera's field of view; Based on the identified amount of dust, the vacuum cleaner of the robot cleaner is controlled to perform cleaning operations on the areas to be cleaned in one or more areas; Acquire first map data indicating the amount of dust in a first area among the one or more areas, wherein the first area corresponds to a portion of the first travel route previously traversed by the robotic cleaner; and Acquire second map data indicating the amount of dust in a second area among the one or more areas, wherein the second area corresponds to the portion of the first travel route that the robotic cleaner has not yet traversed. The control of the vacuum cleaner includes: The area to be cleaned is identified based on the amount of dust in the area; Since the area to be cleaned is not located on the first driving route, a second driving route including the area to be cleaned is established; and Control the drive of the robotic cleaner to make the robotic cleaner move along the second travel path.

11. The method of claim 10, wherein, Controlling the vacuum cleaner also includes: The area in which the amount of dust in one or more of the regions is greater than or equal to a predetermined value is identified as the area to be cleaned.

12. The method of claim 10, further comprising: Based on the fact that the robotic cleaner has completed the cleaning operation along the second travel path, the actuator is controlled to return the robotic cleaner to the first travel path. The acquisition of the second map data includes: acquiring the second map data before performing the cleaning operation, and The acquisition of the first map data includes: re-identifying the amount of dust in the areas previously cleaned by the robotic cleaner in one or more of the regions.

13. The method of claim 12, further comprising: The dust reduction rate of the area to be cleaned is identified based on the first map data and the second map data. as well as The first map data and the dust reduction rate of the identified area to be cleaned are sent to the user terminal device via a communication interface.

14. The method of claim 12, further comprising: Based on the completion of the robot cleaner's journey along the first travel route: Based on the first map data, at least one area to be re-cleaned is identified in one or more of the areas, wherein the at least one area to be re-cleaned has been previously cleaned by a robotic cleaner; Set a third driving route including the at least one area to be re-cleaned; and The actuator is controlled to move the robotic cleaner along a third travel path.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method for controlling a robotic cleaner, the method comprising: As the robotic cleaner travels along the first travel route, at least one image is acquired through the camera of the robotic cleaner. Based on the at least one image, identify the amount of dust in one or more areas within the camera's field of view; Based on the identified amount of dust, the vacuum cleaner of the robot cleaner is controlled to perform cleaning operations on the areas to be cleaned in one or more areas; Acquire first map data indicating the amount of dust in a first area among the one or more areas, wherein the first area corresponds to the portion of the first travel route already traveled by the robotic cleaner; and Acquire second map data indicating the amount of dust in a second area among the one or more areas, wherein the second area corresponds to the portion of the first travel route that the robotic cleaner has not yet traversed. The control of the vacuum cleaner includes: The area to be cleaned is identified based on the amount of dust in the area; Since the area to be cleaned is not located on the first driving route, a second driving route including the area to be cleaned is established; and Control the drive of the robotic cleaner to make the robotic cleaner move along the second travel path.

Images