Robot cleaner and control method therefor

Abstract

A robot cleaner is disclosed. The robot cleaner comprises: a camera; at least one sensor; a memory for storing at least one instruction; and at least one processor for executing the at least one instruction, wherein, when executed individually or collectively by the at least one processor, the at least one instruction instructs the robot cleaner to: identify a first space on the basis of an image generated by the camera or a sensing value generated by the at least one sensor; on the basis of preset unit cells constituting the first space, set, as a second space, a region including at least one cell that does not deviate from an angle-of-view range of the camera; divide the second space into a plurality of regions by grouping the cells; and provide a cleaning guide for the second space on the basis of the plurality of regions.

Description

Robot vacuum cleaner and control method thereof

[0001] The present disclosure relates to a robot vacuum cleaner providing a cleaning guide and a control method thereof.

[0002] Advances in electronic technology have led to the emergence of a variety of electronic devices used in homes. These devices include robot vacuum cleaners.

[0003] For example, there may be a robotic vacuum cleaner that can freely move around within a user's space and perform cleaning.

[0004] A robot vacuum cleaner can refer to an electronic device that automatically performs floor cleaning actions.

[0005] Embodiments of the present disclosure may address at least one of the aforementioned problems and / or disadvantages and provide the advantages described below. Accordingly, the embodiments of the present disclosure provide a robot vacuum cleaner that provides a cleaning guide and a control method thereof.

[0006] Additional embodiments will be presented in the detailed description that follows, some of which will be apparent from the detailed description, and others that may also be learned from the presented embodiments.

[0007] According to at least one embodiment of the present disclosure, a robot cleaner includes a camera, at least one sensor, a memory storing at least one instruction, and at least one processor executing the at least one instruction, wherein the at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to identify a first space based on an image by the camera or a sensing value by the at least one sensor, set an area including at least one cell that does not exceed the field of view range of the camera as a second space based on a preset unit of cells constituting the first space, group the cells to divide the second space into a plurality of areas, and provide a cleaning guide for the second space based on the plurality of areas.

[0008] The above plurality of areas may include at least one of a first area that cannot be cleaned by the robot cleaner and a second area where cleaning has not been completed.

[0009] The robot cleaner further includes a projector, and the at least one instruction, when individually or collectively executed by the at least one processor, can cause the robot cleaner to set a second space including a field of view range of the camera and at least one cell that does not exceed the field of view of the camera.

[0010] The at least one instruction, when executed individually or collectively by the at least one processor, enables the robot cleaner to divide the second space into a plurality of regions based on at least one of the obstacles or contaminations when at least one of the obstacles or contaminations is identified within the second space based on an image from the camera or a sensing value from at least one sensor.

[0011] The present robot cleaner further includes a driving unit, and the at least one instruction, when individually or collectively executed by the at least one processor, controls the driving unit so that the robot cleaner moves to a space outside the second space when cleaning of the second space based on the plurality of areas is completed, identifies a third space based on an image by the camera or a sensing value by at least one sensor, sets an area including at least one cell that does not exceed the field of view of the camera based on a preset unit of cells constituting the third space as a fourth space, groups the cells to divide the fourth space into a plurality of areas, and provides a cleaning guide for the fourth space based on the plurality of areas.

[0012] The robot cleaner further includes a projector, and the at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to include an image projected through the projector, wherein the cleaning guide.

[0013] The at least one instruction, when executed individually or collectively by the at least one processor, enables the robot cleaner to change and project an image of the second area that has been cleaned in real time when cleaning of the second area is completed.

[0014] The robot cleaner further includes a speaker, and the at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to output an audio signal through the cleaning guide.

[0015] The robot cleaner further includes a communication unit, and the at least one instruction, when individually or collectively executed by the at least one processor, enables the robot cleaner to transmit information about the second space divided into the plurality of areas or information about a cleaning guide for the second space to an external electronic device through the communication unit.

[0016] A method for controlling a robot cleaner providing a cleaning guide according to an embodiment of the present disclosure includes a step of identifying a first space based on an image captured by a camera of the robot cleaner or a value sensed by the robot cleaner, a step of setting an area including at least one cell that does not exceed the field of view range of the camera as a second space based on a preset unit of cells constituting the first space, a step of grouping the cells to divide the second space into a plurality of areas, and a step of providing a cleaning guide for the second space based on the plurality of areas.

[0017] The above plurality of areas may include at least one of a first area that cannot be cleaned by the robot cleaner and a second area where cleaning has not been completed.

[0018] The step of setting the second space may include a step of setting an area including at least one cell that does not exceed the image projection range of the projector of the robot cleaner and the angle of view of the camera as the second space.

[0019] The step of dividing the second space into a plurality of regions may be performed by dividing the second space into a plurality of regions based on at least one of an obstacle or contamination identified within the second space based on an image captured by the robot cleaner or a value sensed by the robot cleaner.

[0020] When cleaning of the second space is completed based on the plurality of areas, the method may further include a step of moving the robot cleaner to a space outside the second space, a step of identifying a third space based on an image captured by the robot cleaner or a value sensed by the robot cleaner, a step of setting an area including at least one cell that does not exceed the field of view range as a fourth space based on cells of preset units constituting the third space, a step of grouping the cells to divide the fourth space into a plurality of areas, and a step of providing a cleaning guide for the fourth space based on the plurality of areas.

[0021] A non-transitory readable recording medium including a program for executing a control method of a robot cleaner according to an embodiment of the present disclosure, the control method includes a step of identifying a first space based on an image captured by a camera of the robot cleaner or a value sensed by the robot cleaner, a step of setting an area including at least one cell that does not exceed the field of view range of the camera as a second space based on a preset unit of cells constituting the first space, a step of grouping the cells to divide the second space into a plurality of areas, and a step of providing a cleaning guide for the second space based on the plurality of areas.

[0022] The above and other aspects, features, and advantages of the embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0023] FIG. 1 is a drawing for explaining the operation of a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0024] FIG. 2 is a drawing for explaining the operation of a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0025] FIG. 3 is a block diagram illustrating a configuration of a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0026] FIG. 4 is a detailed block diagram illustrating a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0027] FIG. 5 is a drawing for explaining an operation of a robot cleaner dividing a second space into a plurality of areas according to at least one embodiment of the present disclosure.

[0028] FIG. 6 is a drawing showing an example in which a robot cleaner according to at least one embodiment of the present disclosure projects an image to provide a cleaning guide for a second space.

[0029] FIG. 7 is a sequence diagram showing the order in which a robot cleaner sets up a second space according to at least one embodiment of the present disclosure.

[0030] FIG. 8 is a sequence diagram showing the order in which a robot cleaner according to at least one embodiment of the present disclosure divides a second space into a plurality of areas.

[0031] FIG. 9 is a sequence diagram showing a sequence in which a robot cleaner according to at least one embodiment of the present disclosure projects an image to provide a cleaning guide for a second space.

[0032] FIG. 10 is a drawing showing an example of a robot cleaner dividing a second space into a plurality of areas based on an obstacle according to at least one embodiment of the present disclosure.

[0033] FIG. 11 is a diagram illustrating an example in which a robot cleaner according to at least one embodiment of the present disclosure transmits information about a cleaning guide to an external electronic device.

[0034] FIG. 12 is a flowchart for explaining a method for controlling a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0035] The terms used in the various embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the terms and the overall content of this disclosure, rather than simply their names.

[0036] It should be understood that the various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but include various modifications, equivalents, or substitutes of the embodiments.

[0037] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0038] The singular form of a noun corresponding to an item may include one or more of said items, unless the relevant context clearly indicates otherwise.

[0039] In this disclosure, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0040] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0041] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0042] Terms such as "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the present disclosure, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0043] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0044] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0045] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0046] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor, excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0047] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concepts of the present disclosure are not limited by the relative sizes or spacings drawn in the attached drawings.

[0048] In this disclosure, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0049] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0050] FIG. 1 is a drawing for explaining the operation of a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0051] A robot cleaner (100) is a device that autonomously drives and cleans within a space. When the robot cleaner (100) is equipped with a communication function, the robot cleaner (100) can communicate with various external devices such as a server device (300), a user terminal device (200), and an electronic device (400). For convenience of explanation, in the present disclosure, electronic devices (200, 300, 400) other than the robot cleaner (100) may also be referred to as external devices.

[0052] A user terminal device (200) may be an electronic device used by a user. In FIG. 1, the user terminal device (200) is illustrated as a smartphone, but the user terminal device (200) may be implemented as various devices such as a tablet PC, laptop PC, PC, TV, kiosk smart watch, or smart gear.

[0053] The server device (300) is a device that performs various operations, such as controlling the operation of the robot cleaner (100) or providing the status of the robot cleaner (100) to the user terminal device (200), by performing communication with the robot cleaner (100) and the user terminal device (200). The server device (300) can be implemented as various computing devices, such as a workstation, a cloud, a data drive, and a data station.

[0054] A user can access a server device (300) through a user terminal device (200) and create his / her own account. For example, a user can access a server device (300) by executing an application installed on the user terminal device (200).

[0055] The server device (300) can store information about the created account. The user can register identification information (e.g., IP address, SSID, device serial number, etc.) of various devices, including the robot cleaner (100), in his / her account. The server device (300) can communicate with the robot cleaner (100) based on the registered identification information, and transmit and receive various signals or data. The user terminal device (200) can display the execution screen when the application is executed. The execution screen can include a control menu for controlling the registered devices, including the robot cleaner (100), and information indicating the status of each device.

[0056] For example, when a robot cleaner (100) completes cleaning of an area requiring cleaning while moving within a space, information about the area where cleaning has been completed can be provided to the user terminal device (200) through the server device (300). The area requiring cleaning may include a first area that cannot be cleaned by the robot cleaner (100) or a second area where cleaning has not been completed.

[0057] The robot cleaner (100) can store information about the user space and continuously monitor the cleaning progress of the user space. The robot cleaner (100) can periodically or intermittently transmit the monitoring results to the user terminal device (200) or server device (300). The user can check the cleaning progress of the user space through the robot cleaner (100) or the user terminal device (200).

[0058] FIG. 2 is a drawing for explaining the operation of a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0059] Referring to FIG. 2, the robot cleaner (100) can identify the first space (30) based on an image from a camera or a sensing value from at least one sensor.

[0060] The robot vacuum cleaner (100) can sense the location within a space, the floor surface, the shape of the space, the location and shape of objects (home appliances, furniture, etc.) within the space, etc., by using various sensors such as a lidar sensor, an infrared sensor, an image sensor, and an ultrasonic sensor.

[0061] Specifically, the robot cleaner (100) can sense the shape, configuration, position, etc. of the manual cleaner (10) using various sensors. For example, the height of the head area or the body area of ​​the manual cleaner (10) can be identified through RGB camera images using machine learning techniques such as deep learning.

[0062] In addition, the robot cleaner (100) can determine the distance from the floor based on the height information of the head of the operating manual cleaner (10) to determine whether cleaning is completed for each area. Here, whether the manual cleaner (10) is operating can be identified through the suction sound of the manual cleaner (10). For example, the robot cleaner (100) can include a microphone, and if the suction sound of the manual cleaner (10) is detected above a certain decibel level through the microphone, it can be identified that the manual cleaner (10) is operating.

[0063] The first space (30) where the robot cleaner (100) is located can be a variety of indoor and outdoor environments, such as a home, office, the interior of a building, a factory, or a government office, and can be a space where a manual cleaner (10) is located. Alternatively, the first space (30) can also refer to a space where the robot cleaner (100) performs cleaning.

[0064] Referring to FIG. 2, the robot cleaner (100) stores information about the path along which it has cleaned while driving on its own within the first space (30), and can also identify the first area or the second area requiring cleaning by utilizing the information about the path along which it has cleaned.

[0065] The robot cleaner (100) can divide the first space (30) into cells of preset units, and can also set cells that do not exceed the field of view range based on the preset cells constituting the first space as the second space (20). The operation of dividing the first space or the second space into preset cells and dividing it into multiple areas is described in detail in FIG. 5.

[0066] Here, "cell" may refer to a small area in the form of a grid that constitutes an image. For example, a cell may refer to a rectangular area that divides an image into small blocks or grids. The size of a cell may be set in various ways, and the size value may be preset in the robot cleaner (100) or an external electronic device.

[0067] Here, "angle of view" refers to the extent of a scene that a camera lens can capture, and can refer to the width of the field of view that can be captured in a particular scene. The angle of view can be determined by the lens' focal length and the size of the image sensor.

[0068] Meanwhile, in FIG. 2, a vacuum cleaner that can be cleaned by a user is referred to as a manual vacuum cleaner, but it can be expressed by various terms such as a handy vacuum cleaner, a cordless vacuum cleaner, a portable vacuum cleaner, a portable vacuum cleaner, and a mini vacuum cleaner.

[0069] Below, the operation of the robot vacuum cleaner (100) will be specifically described along with a specific configuration example.

[0070] FIG. 3 is a block diagram illustrating a configuration of a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0071] Referring to FIG. 3, the robot vacuum cleaner (100) includes a camera (110), at least one sensor (120), memory (130), and at least one processor (140).

[0072] The camera (110) is configured to capture images of the surroundings. In FIG. 3, one camera (110) is illustrated, but the camera (110) may include multiple cameras such as a stereo camera, a 3D camera, a TOF (Time of Flight) camera, a depth camera, a multi-lens array camera, a stereo vision system, a fused lidar camera, etc. Among these, a 3D camera is a camera used to capture and create three-dimensional images or videos. Unlike a general 2D camera, it captures images by including depth information, and can accurately determine the distance and spatial position of an object through the depth information, and is utilized in various application fields.

[0073] At least one sensor (120) is a sensor for detecting the surrounding environment. Specifically, it may include at least one or more of a LiDAR sensor, a vision sensor, an image sensor, an infrared sensor, an ultrasonic sensor, a gyro sensor, an acceleration sensor, and a proximity sensor.

[0074] The sensing value of at least one sensor (120) may be provided to a processor (140), etc. The processor (140) may detect the distance from a wall or obstacle in the space to be cleaned, the level of contamination of the floor, an external electronic device, etc., based on the sensing value of at least one sensor (120).

[0075] When at least one sensor (120) includes a 3D LIDAR (Light Detection and Ranging) sensor, the 3D LIDAR sensor can irradiate light, for example, a laser, to a surrounding object and then receive light reflected from the object. The processor (140) can analyze the received light to identify physical properties of the surrounding object, for example, distance, direction, speed, temperature, material distribution, and concentration characteristics.

[0076] According to an embodiment, the memory (130) may store data required for various embodiments of the present disclosure. Depending on the purpose of data storage, the memory (130) may be implemented as a memory embedded in the robot cleaner (100) or may be implemented as a memory that can be attached or detached to the robot cleaner (100).

[0077] For example, data for driving a robot cleaner (100) may be stored in a memory embedded in the robot cleaner (100), and data for extended functions of the robot cleaner (100) may be stored in a memory that can be attached or detached to the robot cleaner (100).

[0078] The memory embedded in the robot cleaner (100) may be implemented in the form of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0079] The memory that can be attached to the robot vacuum cleaner (100) can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), an external memory that can be connected to a USB port (e.g., USB memory), etc.

[0080] The memory (130) may include various instructions necessary for the operation of the processor (140). Here, the instructions may include an instruction for identifying a first space, an instruction for setting cells that do not exceed the field of view of a camera as a second space, an instruction for dividing the second space into a plurality of areas, an instruction for providing a cleaning guide for the second space based on the plurality of areas, an instruction for controlling the movement of the robot cleaner (100), etc.

[0081] The memory (130) can store map information. Here, the map information is information about the space where the robot cleaner (100) is located, and may be information about a space divided into multiple areas. For example, if the robot cleaner (100) is located in a home, the map information may be generated based on a floor plan of the home. This map information may be generated directly by the robot cleaner (100), or may be received and utilized from another electronic device (e.g., a portable projector).

[0082] At this time, the memory (130) can store additional information for each space using information within the map information. Furthermore, the memory (130) can store information about the path along which the robot cleaner has performed cleaning. For example, the robot cleaner (100) can identify cleanliness on a cell-by-cell basis, group cells with a cleanliness level above a preset level, and identify or demarcate them as areas where cleaning has been completed.

[0083] Additionally, the memory (130) may store information about a second space divided into multiple areas or information about a cleaning guide for the second space. Here, the multiple areas may include a first area that cannot be cleaned by the robot vacuum cleaner, a second area that has not been cleaned, an area that has been cleaned, etc.

[0084] According to an embodiment, at least one processor (140) controls the overall operation of the robot cleaner (100). Specifically, at least one processor (140) is connected to each component of the robot cleaner (100) and can control the overall operation of the robot cleaner (100).

[0085] According to an embodiment, at least one processor (140) can control a user terminal device (200) and an electronic device (400).

[0086] At least one processor (140) can perform operations of the robot cleaner (100) according to various embodiments by executing at least one instruction stored in memory.

[0087] At least one processor (140) may set an area including at least one cell that does not exceed the field of view of the camera as a second space based on the preset unit cells that constitute the first space. For example, at least one processor (140) may set an area including at least one cell that does not exceed the image projection range of the projector and the field of view of the camera as the second space.

[0088] And at least one processor (140) can group cells to partition the second space into multiple areas.

[0089] For example, at least one processor (140) may divide a first space into cells of preset units based on images captured by a camera or sensing values ​​captured by at least one sensor. At least one processor (140) may identify the cleanliness (or contamination) of each divided cell and group the cells into multiple areas, such as a cleaned area, an ungroomed area, and a re-cleaned area.

[0090] Here, various image processing techniques such as labeling and morphology operations can be used as grouping methods.

[0091] Here, the term "re-cleaning area" may refer to a second area that has not yet been cleaned. The term "uncleaned area" may refer to a first area that cannot be cleaned by a robot vacuum cleaner. The term "cleaned area" may include an area that has been cleaned by a robot vacuum cleaner or an area that has been cleaned by a manual vacuum cleaner.

[0092] At least one processor (140) may provide a cleaning guide for a second space based on multiple areas. Here, the cleaning guide may provide information about the first area or the second area requiring cleaning in the form of a video or audio.

[0093] At least one processor (140) may control the projector (160) to change and project an image of the first or second area that has been cleaned in real time when cleaning of the first or second area is completed. The operation of the robot cleaner (100) to provide a cleaning guide by reflecting the cleaning status in real time is described in detail in FIG. 6.

[0094] According to an embodiment, at least one processor (140) may be implemented as a digital signal processor (DSP), a microprocessor, or a timing controller (TCON) that processes a digital signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an artificial intelligence (AI) processor, or may be defined by the relevant terms. In addition, at least one processor (140) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). At least one processor (140) may perform various functions by executing computer executable instructions stored in a memory.

[0095] For example, at least one processor (140) may include one or more processing circuits. And at least one processor (140) may include one or more processing circuits configured to individually and / or collectively perform various functions of the present disclosure. As a non-limiting example, at least a portion of the processor (140) may be included in a first chip of the robot cleaner (100), and at least another portion of the processor (140) may be included in a second chip of a robot cleaner (or electronic device) that is different from the first chip of the robot cleaner (100).

[0096] For example, at least one processor (140) may include one or more of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a DSP (Digital Signal Processor), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator.

[0097] At least one processor (140) may control one or any combination of other components of the electronic device and perform operations related to communication or data processing. At least one processor (140) may execute one or more programs or instructions stored in memory. For example, at least one processor (140) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in memory.

[0098] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors.

[0099] For example, when the first operation, the second operation, and the third operation are performed by the method according to the embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-only processor).

[0100] At least one processor (140) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore).

[0101] When at least one processor (140) is implemented as a multi-core processor, each of the plurality of cores included in the multi-core processor may include internal processor memory such as cache memory and on-chip memory, and a common cache shared by the plurality of cores may be included in the multi-core processor.

[0102] Each of the plurality of cores (or some of the plurality of cores) included in the multi-core processor may independently read and execute program instructions for implementing a method according to an embodiment of the present disclosure, or all (or some) of the plurality of cores may be linked to read and execute program instructions for implementing a method according to an embodiment of the present disclosure.

[0103] FIG. 4 is a detailed block diagram illustrating a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0104] Referring to FIG. 4, a robot cleaner (100) according to an embodiment of the present disclosure may include a camera (110), at least one sensor (120), a memory (130), at least one processor (140), a driving unit (150), a projector (160), a speaker (170), and a communication unit (180). Parts that overlap with the above description will be omitted or abbreviated.

[0105] The driving unit (150) is a component for moving the main body of the robot cleaner (100). The driving unit (150) may include components such as a plurality of wheels, a driving motor for rotating each of the plurality of wheels, a gear, and a shaft. The plurality of wheels are provided on the lower or side of the main body of the robot cleaner (100) and support the main body of the robot cleaner (100) from the floor surface. When the driving motor operates and the driving force is transmitted to the plurality of wheels so that each wheel rotates, the robot cleaner (100) can move by the frictional force between the floor surface and the wheels. In addition, the driving unit (150) may adjust the rotational speed of at least one wheel among the plurality of wheels differently or the alignment direction of the wheels differently when changing direction. Depending on the roughness of the floor surface, frictional force, etc., an infinite track may be used instead of the wheels.

[0106] For example, when cleaning of a second space is completed based on multiple areas, the processor (140) may control the driving unit (150) to move the robot cleaner (100) to a space outside the second space. When the robot cleaner (100) leaves the second space, it may repeat the operations performed in the first space and the second space.

[0107] For example, the processor (140) may identify a third space based on an image by a camera (110) or a sensing value by at least one sensor (120), and may set cells that do not exceed the field of view range of the camera (110) as a fourth space based on a preset unit of cells that constitute the third space.

[0108] Additionally, the processor (140) can group cells to divide the fourth space into multiple areas and provide a cleaning guide for the fourth space based on the multiple areas.

[0109] Here, the third space may mean a space outside the second space that was the target of the cleaning guide, and the fourth space may mean a space reduced based on the angle of view of the camera (110) in the third space.

[0110] However, it is not limited thereto, and the robot cleaner (100) may repeat the above-described operation until cleaning of the user space is completed outside the fourth space.

[0111] The projector (160) can project a cleaning guide for a space requiring cleaning onto the floor. For example, the robot cleaner (100) can project images for each of the multiple areas through the projector (160) in a second space divided into multiple areas.

[0112] According to one embodiment, the projector (160) may include a light source (not shown). The light source (not shown) may include, but is not limited to, an LED, a laser, and a lamp. In addition, according to one embodiment, the image projector (160) may include a convex lens (not shown) together with the light source (not shown). In addition, according to one embodiment, when the cooking information image is a virtual moving guide image, the image projector (160) may include a light source (not shown), a convex lens (not shown), an air injection unit (not shown), and a guide member (not shown).

[0113] In addition, according to one embodiment, the projector (160) may include an LCD (Liquid Crystal Display) projector, a DLP (Digital Light Processing) projector, an LED (Light-emitting diode) projector, an LCoS (Liquid Crystal on Silicon) and a laser projector. However, the present invention is not limited thereto, and as illustrated in FIG. 11, information about the cleaning guide may be transmitted to an external electronic device to provide the cleaning guide through the projector of the external electronic device.

[0114] The speaker (170) is configured to provide information about the cleaning guide. The speaker (170) is provided in at least one of the upper area, lower area, and side area of ​​the robot cleaner (100) and can provide various information, such as various auditory feedback or information about the cleaning guide, through audio.

[0115] For example, a robot cleaner (100) may output audio signals through a speaker (170) for each of a plurality of areas in a second space divided into multiple areas to provide a cleaning guide for the second space. In this case, even without a projector, the robot cleaner (100) may control the speaker (170) to provide a cleaning guide using only sound.

[0116] The communication unit (180) can communicate with an external server or external electronic device. In particular, the communication unit (180) can also receive map information about the user space from an external server or external electronic device. The operation of providing a cleaning guide by communicating with an external device is described in detail in FIG. 11.

[0117] The communication unit (180) may include wired or wireless input / output interfaces (or input / output terminals) according to various standards. For example, one or more connection interfaces may include various interfaces such as HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface), AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc.

[0118] FIG. 5 is a drawing for explaining an operation of a robot cleaner dividing a second space into a plurality of areas according to at least one embodiment of the present disclosure.

[0119] The processor (140) controls the driving wheels according to the driving path to move the main body, and divides the space where the robot cleaner (100) is located into cells of preset units, and groups the cells to be divided to form multiple areas. While the robot cleaner (100) is driving, at least one sensor (120) can continuously measure the distance to surrounding objects, the level of contamination of the floor, the depth of objects, etc.

[0120] For example, when a robot cleaner (100) identifies a first space (30) based on an image captured by a camera (110) or a sensing value captured by at least one sensor (120), the first space can be divided into cells of preset units.

[0121] The processor (140) can set cells that do not exceed the field of view range of the camera (110) based on the cells that constitute the first space as the second space (20). The operation of setting the second space (20) based on the cells is described in detail in FIG. 7.

[0122] The processor (140) can group the cells constituting the second space (20) and divide them into multiple regions. For example, the cells can be divided into an ungroomed region (21), a re-cleaned region (22), and a cleaned region (23) depending on whether cleaning has been performed or the level of cleanliness of each cell constituting the second space (20).

[0123] Here, whether cleaning of each cell is completed can be determined based on the contamination level of each cell or whether cleaning is performed by the cleaner, but is not limited thereto.

[0124] FIG. 6 is a drawing showing an example in which a robot cleaner according to at least one embodiment of the present disclosure projects an image to provide a cleaning guide for a second space.

[0125] The processor (140) can divide the second space (20) into a first area (21-1, 21-2) that cannot be cleaned by the robot cleaner (100) and a second area (22) that has not been cleaned, and project an image to provide a cleaning guide for each area.

[0126] In this case, the layout, shape, size, etc. of the video providing the cleaning guide may be changed in various ways.

[0127] When a user performs cleaning of the second area (22) according to the cleaning guide using a manual cleaner (10), the robot cleaner (100) can reflect the cleaning status in real time and change the image displayed as an area requiring cleaning to display a cleaning completed area (23). The operation of identifying whether cleaning has been performed for each area is described in detail in Fig. 9.

[0128] In this case, the robot cleaner (100) may provide a cleaning guide by displaying areas that have not been cleaned until cleaning is completed.

[0129] However, it is not limited to this, and the robot cleaner (100) can also control the speaker (170) to provide a cleaning guide in real time in the form of audio.

[0130] FIG. 7 is a sequence diagram showing the order in which a robot cleaner sets up a second space according to at least one embodiment of the present disclosure.

[0131] Referring to FIG. 7, the robot cleaner (100) can detect and expand the first space in cell units (S710).

[0132] The robot cleaner (100) can divide cells that are outside the projection range of the projector (160) among the cells forming the first space and set them as new spaces (S720). Similarly, the robot cleaner (100) can divide cells that are outside the viewing angle range of the camera (110) and set them as new spaces (S730).

[0133] That is, the robot cleaner (100) can identify whether grouping into multiple areas is complete based on cells that do not exceed the field of view of the camera (110) and cells that do not exceed the image projection range of the projector (160) (S740). Here, if grouping into multiple areas is not complete, the robot cleaner can repeatedly perform the operation of detecting and expanding the first space in units of cells.

[0134] If the robot cleaner (100) has completed grouping into multiple areas, it can set a second space based on the grouped multiple areas (S750).

[0135] FIG. 8 is a sequence diagram showing the order in which a robot cleaner according to at least one embodiment of the present disclosure divides a second space into a plurality of areas.

[0136] Referring to FIG. 8, the robot cleaner (100) can generate a map for the user space based on an image captured by a camera (110) and a sensing value of at least one sensor (120) (S810).

[0137] For example, a robot cleaner (100) can create a map of the user's space using a LIDAR sensor and SLAM technology. However, this is not limited to this, and the robot cleaner (100) can also create a map using various sensors or various technologies.

[0138] The robot cleaner (100) can perform a driving operation while performing cleaning by controlling the driving unit (150) (S820). If cleaning is performed normally in a specific area, the robot cleaner (100) can mark the area where cleaning was performed normally as a cleaning completed area (S830).

[0139] If cleaning is not performed normally and at least one obstacle or contamination is detected within the space, the robot cleaner (100) can identify the area where the obstacle or contamination is detected as an area that cannot be cleaned and display it as an ungroomed area (S840).

[0140] If cleaning is not performed normally and no obstacles are detected in the space, but contamination is detected, the robot cleaner (100) can identify the area where contamination is detected as an area requiring re-cleaning and display it as a re-cleaning area (S850).

[0141] The robot vacuum cleaner (100) can divide the space into multiple areas such as an ungroomed area, a re-cleaned area, and a cleaned area while moving through the user's space (S860).

[0142] FIG. 9 is a sequence diagram showing a sequence in which a robot cleaner according to at least one embodiment of the present disclosure projects an image to provide a cleaning guide for a second space.

[0143] Referring to FIG. 9, the robot cleaner (100) may initiate a cleaning guide for a second space based on a plurality of areas (S910). The robot cleaner (100) may group cells constituting the second space into a plurality of areas and project an image related to the cleaning guide based on the divided plurality of areas (S920). For example, if the robot cleaner is equipped with a projector, the cleaning guide may include an image projected through the projector. Alternatively, if the robot cleaner includes a speaker, the cleaning guide may include an audio signal output through the speaker.

[0144] The robot cleaner (100) can track the head movement of a manual cleaner performing cleaning by controlling a camera (110) or at least one sensor (120) (S930). The robot cleaner (100) can also mark an area where the head has passed as a cleaning completion area (S940). In this case, even if the head has passed through an area, if the manual cleaner does not output a cleaning sound, the area may not be marked as a cleaning completion area. In other words, the cleaning completion area can be identified based on the head movement and the cleaning sound output from the manual cleaner.

[0145] In this case, the robot cleaner (100) may use an RGB camera or LADAR sensor to map the position of the head onto a map of space.

[0146] The robot cleaner (100) can identify a cleaned area in real time, update the image of the cleaned area, and project it onto the floor (S950). That is, when cleaning of the first or second area requiring cleaning is completed, the robot cleaner (100) can change and project the image of the first or second area that has been cleaned in real time.

[0147] When the robot cleaner (100) completes cleaning of multiple areas constituting the second space (S960), it can move to a space outside the second space and repeat the above-described operation.

[0148] FIG. 10 is a drawing showing an example of a robot cleaner dividing a second space into a plurality of areas based on an obstacle according to at least one embodiment of the present disclosure.

[0149] Referring to Fig. 10, an obstacle (11) may be located in an ungroomed area (21) among multiple areas. In this case, the robot cleaner may divide the ungroomed area (21) into multiple areas based on the location of the obstacle (11). The robot cleaner (100) may also identify an obstacle (11) within a space by identifying whether the obstacle is larger than a preset size.

[0150] For example, the robot cleaner (100) can divide the ungroomed area (21) into ungroomed area 1 (21-1) and ungroomed area 2 (21-2) by dividing the ungroomed area (21) into two halves based on an arbitrary straight line passing through the center of the obstacle (11).

[0151] In this case, the robot cleaner (100) can move to a position to provide a cleaning guide for the ungroomed area 1 (21-1), and when cleaning of the ungroomed area 1 (21-1) is completed, the robot cleaner (100) can move to a position to provide a cleaning guide for the ungroomed area 2 (21-2).

[0152] In Fig. 10, the uncleaned area 1 (21-1) and the uncleaned area 2 (21-2) are depicted as having the same size, but this is not limited to this and they may be divided into different sizes.

[0153] The robot cleaner (100) can be positioned so that it can move horizontally from an arbitrary straight line passing through the center of the obstacle (11) from a position facing the obstacle (11) in order to avoid and project the obstacle (11).

[0154] FIG. 11 is a diagram illustrating an example in which a robot cleaner according to at least one embodiment of the present disclosure transmits information about a cleaning guide to an external electronic device.

[0155] Referring to FIG. 11, the robot cleaner (100) may also provide a cleaning guide using a projector of an external electronic device (400).

[0156] Specifically, the robot cleaner (100) can transmit information about a second space divided into multiple areas or information about a cleaning guide for the second space to an external electronic device through a communication unit (180).

[0157] In this case, an external electronic device (400), such as a mobile projector, can project an image onto the floor for the second space (20) set by the robot cleaner (100) to provide a cleaning guide.

[0158] However, it is not limited thereto, and the external electronic device (400) can provide a cleaning guide reflecting the cleaning status in real time, and can also provide the cleaning guide in the form of audio through the speaker of the external electronic device (400).

[0159] FIG. 12 is a flowchart for explaining a method for controlling a robot vacuum cleaner according to at least one embodiment of the present disclosure.

[0160] According to FIG. 12, the robot cleaner identifies a first space based on images captured by the robot cleaner or values ​​sensed by the robot cleaner while driving within the space (S1210). The first space may refer to a space in which the robot cleaner can drive.

[0161] The robot cleaner sets cells that do not exceed the field of view range as a second space based on the preset unit cells that constitute the first space (S1220). The field of view range may refer to the field of view range of a camera included in the robot cleaner.

[0162] The robot vacuum cleaner groups cells to divide the second space into multiple areas (S1230). The multiple areas may include areas where cleaning has not been performed, areas where cleaning has been performed but not yet completed, and areas where cleaning has been completed.

[0163] The robot vacuum cleaner provides a cleaning guide for a second space based on multiple areas (S1240). Here, the cleaning guide can be provided through various methods, such as projecting an image or outputting an audio signal.

[0164] The image generation method described in Fig. 12 can be performed by devices having various configurations such as those of Figs. 3 and 4 described above, but is not necessarily limited thereto, and can also be performed by devices having various configurations.

[0165] The various embodiments described above may be implemented as a single embodiment, or at least one of the embodiments may be combined in whole or in part to be implemented together in one device.

[0166] According to the various embodiments described above, the user can clean more effectively by defining the area requiring cleaning in more detail and providing real-time information about the cleaning area. Ultimately, this can enhance the user experience.

[0167] Various embodiments of the present disclosure may be implemented as software stored in a machine-readable storage media that can be installed or connected to a smartphone, a user terminal device, or other various electronic devices (e.g., a computer).

[0168] Specifically, a non-transitory readable storage medium may be provided in which software is stored for sequentially performing the steps of: identifying a first space based on an image captured by a robot cleaner or a value sensed by the robot cleaner; setting cells that do not exceed the field of view range as a second space based on preset unit cells constituting the first space; grouping the cells to divide the second space into a plurality of areas; and providing a cleaning guide for the second space based on the plurality of areas.

[0169] A device equipped with such a non-transitory readable medium can perform various operations, such as identifying a first space, setting a second space, dividing the second space into multiple areas, and providing a cleaning guide for the second space, as described in the various embodiments described above.

[0170] In the context of non-transitory readable storage media, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily on the storage medium.

[0171] Alternatively, a program for performing the method according to the various embodiments described above may be distributed online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0172] Each component (e.g., a module or a program) according to various embodiments may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., a module or a program) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration.

[0173] According to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0174] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In robot vacuum cleaners, camera; At least one sensor; memory that stores at least one instruction; and At least one processor executing at least one instruction; The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: Identifying a first space based on an image from the camera or a sensing value from at least one sensor, Based on the preset unit cells that constitute the first space, an area including at least one cell that does not exceed the field of view range of the camera is set as a second space, By grouping the above cells, the second space is divided into multiple areas, A robot vacuum cleaner that provides a cleaning guide for the second space based on the plurality of areas.

2. In paragraph 1, The above multiple areas are, A robot vacuum cleaner comprising at least one of a first area that cannot be cleaned by the robot vacuum cleaner and a second area that has not been cleaned.

3. In paragraph 1, Including a projector; The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: A robot vacuum cleaner, which is set to a second space including the field of view range of the camera and at least one cell that does not exceed the field of view of the camera.

4. In paragraph 1, The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: A robot cleaner that divides the second space into a plurality of areas based on at least one of the obstacles or contaminations when at least one of the obstacles or contaminations is identified within the second space based on an image from the camera or a sensing value from at least one sensor.

5. In paragraph 1, Including the driving section; The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: When cleaning of the second space is completed based on the plurality of areas, the driving unit is controlled so that the robot cleaner moves to a space outside the second space, Identifying a third space based on an image from the camera or a sensing value from at least one sensor, Based on the preset unit cells that constitute the third space, an area including at least one cell that does not exceed the angle of view of the camera is set as a fourth space, By grouping the above cells, the fourth space is divided into multiple areas, A robot vacuum cleaner that provides a cleaning guide for the fourth space based on the plurality of areas.

6. In paragraph 1, Including a projector; The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: A robot cleaner, wherein the above cleaning guide includes an image projected through the above projector.

7. In paragraph 6, The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: A robot vacuum cleaner that changes and projects an image of the second area that has been cleaned in real time when cleaning of the second area is completed.

8. In paragraph 1, including speakers; The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: A robot vacuum cleaner, wherein the cleaning guide includes an audio signal output through the speaker.

9. In paragraph 1, Including the Department of Communications; The at least one instruction, when individually or collectively executed by the at least one processor, causes the robot cleaner to: A robot vacuum cleaner that transmits information about the second space divided into the plurality of areas or information about a cleaning guide for the second space to an external electronic device through the communication unit.

10. In a control method of a robot vacuum cleaner providing a cleaning guide, A step of identifying a first space based on an image captured by a camera of the robot cleaner or a value sensed by the robot cleaner; A step of setting an area including at least one cell that does not exceed the field of view range of the camera as a second space based on the preset unit cells that constitute the first space; A step of dividing the second space into multiple areas by grouping the cells; and A control method comprising: a step of providing a cleaning guide for the second space based on the plurality of areas; 11. In paragraph 10, The above multiple areas are, A control method comprising at least one of a first area that cannot be cleaned by the robot vacuum cleaner and a second area where cleaning has not been completed.

12. In paragraph 10, The step of setting the above second space is: A control method comprising the step of setting an area including at least one cell that does not exceed the image projection range of the projector of the robot cleaner and the angle of view of the camera as a second space.

13. In paragraph 10, The step of dividing the above second space into multiple areas is: A control method for dividing the second space into a plurality of areas based on at least one of an obstacle or contamination identified within the second space based on an image captured by the robot cleaner or a value sensed by the robot cleaner.

14. In paragraph 10, When cleaning of the second space is completed based on the plurality of areas, the robot cleaner moves to a space outside the second space; A step of identifying a third space based on an image captured by the robot cleaner or a value sensed by the robot cleaner; A step of setting an area including at least one cell that does not exceed the field of view range based on the preset unit cells constituting the third space as a fourth space; A step of dividing the fourth space into multiple areas by grouping the cells; and A control method further comprising: a step of providing a cleaning guide for the fourth space based on the plurality of areas; 15. In a non-transitory readable recording medium including a program for executing a control method of a robot vacuum cleaner, The above control method is, A step of identifying a first space based on an image captured by a camera of the robot cleaner or a value sensed by the robot cleaner; A step of setting an area including at least one cell that does not exceed the field of view range of the camera as a second space based on the preset unit cells that constitute the first space; A step of dividing the second space into multiple areas by grouping the cells; and A non-transitory readable recording medium comprising a step of providing a cleaning guide for the second space based on the plurality of areas.

Images