Method for controlling cleaning robot to avoid liquid and cleaning robot using same

Abstract

A cleaning robot that avoids liquid during travel and a method of controlling the cleaning robot are provided. The cleaning robot can control a light emitting unit to emit infrared light to a detection area on a to-be-cleaned surface in front of the cleaning robot; receiving infrared light reflected from the detection area through an infrared light sensor, and receiving visible light reflected from the detection area through a visible light sensor; based on the intensity of the received infrared light and the visible light, whether liquid exists on the surface to be cleaned or not is determined; moreover, based on determining that the liquid exists on the to-be-cleaned surface, the advancing module is controlled, so that the cleaning robot avoids the liquid in the advancing process.

Description

Technical Field

[0001] The present disclosure relates to a cleaning robot and an operation method thereof. More specifically, the present disclosure relates to a method for a cleaning robot to detect a liquid object on a surface to be cleaned and move forward while avoiding the liquid object, and a cleaning robot for performing the method. Background Art

[0002] Recently, robots that can move while automatically cleaning the floor and avoiding obstacles (i.e., autonomous cleaning robots, hereinafter also simply referred to as cleaning robots) have attracted attention. General cleaning robots clean by sweeping up dust with a brush and sucking it in, but cleaning robots that perform cleaning by wiping dirt with a mop or the like, or cleaning robots that combine these cleaning methods, have also been put into practical use.

[0003] With the progress of computer technology, cleaning robots can detect indoor obstacles, such as furniture, household appliances, or interior decorations, with relatively high accuracy and avoid these obstacles while traveling on the floor. Summary of the Invention

[0004] One aspect of an embodiment of the present disclosure may provide a cleaning robot that avoids a liquid object while traveling. The cleaning robot may include a cleaning module, a traveling module configured to move the cleaning robot on a surface to be cleaned, a light emitting unit, an infrared light sensor, a visible light sensor, at least one memory storing one or more instructions, and at least one processor. In addition, the at least one processor may execute the one or more instructions stored in the memory to control the light emitting unit to emit infrared light to a detection area on the surface to be cleaned in front of the cleaning robot. In addition, the at least one processor may receive infrared light reflected from the detection area through the infrared light sensor. In addition, the at least one processor may receive visible light reflected from the detection area through the visible light sensor. In addition, the at least one processor may determine whether there is a liquid object in the detection area based on the intensity of the received infrared light and the visible light. In addition, based on determining that there is a liquid object in the detection area in front of the cleaning robot, the at least one processor may control the traveling module to move the cleaning robot to avoid the liquid object.

[0005] One aspect of embodiments of the present disclosure may provide a method for a cleaning robot to avoid liquid objects. The cleaning robot may emit infrared light to a detection area on a surface to be cleaned in front of the cleaning robot. In addition, the cleaning robot may receive the infrared light from the surface to be cleaned through an infrared light sensor. In addition, the cleaning robot may receive visible light reflected from the surface to be cleaned through a visible light sensor. In addition, the cleaning robot may determine whether there is a liquid object in the detection area based on the intensity of the received infrared light and the visible light. In addition, based on determining that there is a liquid object on the surface to be cleaned, the cleaning robot may move the cleaning robot to avoid the liquid object. Description of the Drawings

[0006] Figure 1 Shows a method for a cleaning robot to avoid liquid objects on a surface to be cleaned according to an embodiment of the present disclosure.

[0007] Figure 2 Shows a block diagram of a cleaning robot according to an embodiment of the present disclosure.

[0008] Figure 3a Shows the lower surface of a cleaning robot according to an embodiment of the present disclosure.

[0009] Figure 3b Shows the front surface of a cleaning robot according to an embodiment of the present disclosure.

[0010] Figure 4 Is a flowchart of a method for a cleaning robot to avoid liquid objects according to an embodiment of the present disclosure.

[0011] Figure 5 Is a flowchart of a method for a cleaning robot to avoid liquid objects based on the light absorption characteristics of water according to an embodiment of the present disclosure.

[0012] Figure 6a Shows an infrared light sensor of a cleaning robot configured to receive infrared light having an absorption wavelength according to an embodiment of the present disclosure.

[0013] Figure 6b Shows the light absorption spectrum of water according to an embodiment of the present disclosure.

[0014] Figure 7 Is a flowchart of a method for a cleaning robot to avoid liquid objects based on the reflectance corresponding to the incident angle according to an embodiment of the present disclosure.

[0015] Figure 8a Shows a method for a cleaning robot to emit infrared light to a liquid object in a detection area according to an embodiment of the present disclosure.

[0016] Figure 8bShows a method for determining the presence or absence of a liquid object based on light reflectance, performed by a cleaning robot according to an embodiment of the present disclosure.

[0017] Figure 9a And Figure 9b Shows the relationship between the incident angle of infrared light and the regular reflection ratio according to an embodiment of the present disclosure.

[0018] Figure 10a Shows a method for determining the presence or absence of a liquid object on a surface to be cleaned by using a machine learning model, performed by a cleaning robot according to an embodiment of the present disclosure.

[0019] Figure 10b Shows a method for determining the presence or absence of a liquid object on a surface to be cleaned by using a machine learning model, performed by a cleaning robot according to an embodiment of the present disclosure.

[0020] Figures 11a to 11c Shows a diagram for describing a method for determining a liquid object on a surface to be cleaned, performed by a cleaning robot according to an embodiment of the present disclosure.

[0021] Figure 11d Shows a diagram for describing a method for determining a liquid object on a surface to be cleaned, performed by a cleaning robot according to an embodiment of the present disclosure.

[0022] Figure 12 Is a flowchart of a method for detecting and avoiding a liquid object during travel, performed by a cleaning robot according to an embodiment of the present disclosure.

[0023] Figure 13 Shows a method for determining the presence or absence of a liquid object by using a machine learning model, performed by a cleaning robot according to an embodiment of the present disclosure.

[0024] Figure 14 Shows a method for determining the presence or absence of a liquid object based on the difference between an infrared light image and a visible light image, performed by a cleaning robot according to an embodiment of the present disclosure.

[0025] Figure 15 Shows a method for notifying the presence of a liquid object in an area to be cleaned, performed by a cleaning robot according to an embodiment of the present disclosure.

[0026] Figure 16 Shows a method for notifying the presence of a liquid object in an area to be cleaned through a user device, performed by a cleaning robot according to an embodiment of the present disclosure.

[0027] Figure 17 Shows a block diagram of a cleaning robot according to an embodiment of the present disclosure. Detailed Description

[0028] As used herein, the expression "at least one of a, b, or c" may represent only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can implement the present disclosure without difficulty. However, the present disclosure can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In addition, to clearly describe the present disclosure, parts not relevant to the description are omitted, and throughout this specification, like reference numerals are assigned to like elements.

[0030] Although the terms used herein are common terms that are currently widely used and selected considering their functions, the meanings of the terms may change according to the intentions of those of ordinary skill in the art, legal precedents, or the emergence of new technologies. Therefore, the terms should not be defined by their simple appellations, but should be defined based on their meanings and the context described in the present disclosure.

[0031] In addition, terms such as "first" or "second" may be used to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0032] In addition, the terms used herein are for describing specific embodiments and are not intended to limit the scope of the present disclosure. Singular expressions also include plural meanings as long as they are not inconsistent with the context. In addition, throughout this specification, when a component is referred to as being "connected to" another component, the component may be "directly connected to" the other component or "electrically connected to" the other component through an intermediate element. In addition, when an element is referred to as "including" a component, the element may additionally include other components as long as there is no specific contrary statement, rather than excluding other components.

[0033] Phrases such as "in some embodiments" or "in an embodiment" used herein do not necessarily refer to the same embodiment.

[0034] Embodiments of the present disclosure provide a cleaning robot for detecting liquid on a surface to be cleaned and a method for controlling the cleaning robot.

[0035] Embodiments of the present disclosure provide a cleaning robot for determining an area of liquid on a surface to be cleaned and a method for controlling the cleaning robot.

[0036] Embodiments of the present disclosure provide a cleaning robot for avoiding liquid on a surface to be cleaned and a method for controlling the cleaning robot.

[0037] Figure 1 Disclosed is a method for a cleaning robot 1000 to avoid a liquid object on a surface to be cleaned according to an embodiment of the present disclosure.

[0038] Referring to Figure 1 , the cleaning robot 1000 can determine whether there is a liquid object W on the surface to be cleaned F based on both infrared light and visible light reflected from the surface to be cleaned F.

[0039] The surface to be cleaned F may refer to the surface that the cleaning robot 1000 is to clean. The cleaning robot 1000 can suck dust or garbage on the surface to be cleaned F while moving on the surface to be cleaned F. The surface to be cleaned F may include, for example, the floor in a building, or a carpet, blanket, or mat on the floor, but is not limited thereto.

[0040] The liquid object W may refer to a substance containing liquid. The liquid may be a freely flowing substance that changes its shape according to the shape of the container and does not have a specific shape, such as water or oil. In addition, the liquid object W may consist only of liquid, such as water, oil, juice, or animal urine, or may be a substance containing liquid, such as wet food or a wet towel. In addition, the liquid object W may be in the form of a gel.

[0041] The light emitting units 1800a and 1800b, the visible light sensor 1720, and the infrared light sensor 1710 may be disposed on the front upper surface of the robot body 2. By using the light emitting units 1800a and 1800b, the visible light sensor 1720, and the infrared light sensor 1710, the cleaning robot 1000 can determine whether there is a liquid object W on the surface to be cleaned F in front of the traveling path.

[0042] According to an embodiment of the present disclosure, the cleaning robot 1000 can determine whether there is a liquid object W on the surface to be cleaned F based on the characteristic that when the incident angle is greater than or equal to the threshold angle, the surface reflectivity of the liquid object W is greater than the surface reflectivity of other objects outside the liquid object W.

[0043] For example, the cleaning robot 1000 can control the light emitting units 1800a and 1800b to emit infrared light to the surface to be cleaned F and receive the infrared light from the surface to be cleaned F through the infrared light sensor 1710. Among the emitted infrared light, the infrared light not received back by the infrared light sensor 1710 can be regarded as the reflected infrared light.

[0044] In addition, the cleaning robot 1000 may calculate the reflectivity of infrared light based on the received infrared light intensity relative to the emitted infrared light intensity. The cleaning robot 1000 may determine that there is a liquid object in an area where the reflectivity is greater than or equal to a threshold reflectivity in the detection area. When determining the presence or absence of a liquid object based on the surface reflectivity of the liquid object W, the light sources of the light emitting units 1800a and 1800b may include ultraviolet light as well as infrared light. Household light sources, such as fluorescent lamps or white light emitting diodes (LEDs), are visible lights with wavelengths of approximately 400 nm to 800 nm and do not include ultraviolet light or infrared light. Therefore, the cleaning robot 1000 can detect the liquid object W by using ultraviolet light or infrared light, thereby suppressing the influence of ambient light. Even when the reflectivity of the liquid object W is low, the liquid object W appears relatively darker in the captured image compared to the area around the liquid object W.

[0045] According to an embodiment of the present disclosure, the cleaning robot 1000 may determine whether there is a liquid object W on the surface F to be cleaned based on the light absorption characteristics of water for light wavelengths.

[0046] For example, the cleaning robot 1000 may control the light emitting units 1800a and 1800b to emit light having an absorption wavelength as a peak wavelength toward the surface F to be cleaned, and receive the light having the absorption wavelength in the light reflected from the surface F to be cleaned through the infrared light sensor 1710. In addition, the cleaning robot 1000 may determine whether there is a liquid object W on the surface F to be cleaned based on the intensity of the received light having the absorption wavelength. The absorption wavelength may refer to a wavelength having a higher light absorption rate than adjacent wavelengths according to the light absorption characteristics of water. In addition, according to an embodiment of the present disclosure, the absorption wavelength may be determined within the range of the near-infrared region. For example, the absorption wavelength may be between 900 nm and 1000 nm.

[0047] The cleaning robot 1000 may control the light emitting units 1800a and 1800b to emit light having an absorption wavelength as a peak wavelength. For example, the cleaning robot 1000 may control the light emitting units 1800a and 1800b to emit light in which the intensity of the light having the absorption wavelength is higher than the intensity of the light of adjacent wavelengths.

[0048] The cleaning robot 1000 may receive the light having the absorption wavelength in the light reflected from the surface F to be cleaned through the infrared light sensor 1710.

[0049] According to an embodiment of the present disclosure, the infrared light sensor 1710 may include a band-pass filter for only receiving the light having the absorption wavelength. Accordingly, the infrared light sensor 1710 may block visible light and receive the light having the absorption wavelength in the near-infrared region.

[0050] When there is a liquid object W on the surface F to be cleaned, the water in the liquid object W can absorb more light with an absorption wavelength than the surrounding area of the liquid object W. Accordingly, the intensity of the light with the absorption wavelength reflected from the liquid object W can be lower than the intensity of the light with the absorption wavelength reflected from the surrounding area of the liquid object W. In addition, when the intensity of the reflected light with the absorption wavelength is converted into an image, the area of the liquid object W may appear darker in the image than the surrounding area of the liquid object W.

[0051] According to an embodiment of the present disclosure, the cleaning robot 1000 may determine an area where the brightness in the converted image is lower than a reference brightness or much lower than the surrounding area as the area of the liquid object W. According to an embodiment of the present disclosure, the cleaning robot 1000 may consider not only infrared light but also visible light received from the surface F to be cleaned, so as to more accurately determine whether there is a liquid object W on the surface F to be cleaned. The cleaning robot 1000 may receive visible light reflected from the surface F to be cleaned through the visible light sensor 1720.

[0052] In the infrared light image based on the infrared light with an absorption wavelength, the area of the liquid object W appears darker than the surrounding area. In addition, in the infrared image based on an incident angle greater than or equal to a threshold angle, the area of the liquid object W also appears darker than the surrounding area. However, in the infrared light image, the area corresponding to the dark part of the surface F to be cleaned or the shadow part on the surface F to be cleaned due to the emitted light also appears dark. When the surface F to be cleaned is dark, or in order to minimize the influence of the shadow due to the emitted light, the cleaning robot 1000 may consider not only the infrared light image but also the visible light image based on the visible light reflected from the surface F to be cleaned.

[0053] According to an embodiment of the present disclosure, the cleaning robot 1000 may identify whether there is a liquid object W on the surface F to be cleaned based on the difference image between the infrared light image and the visible light image. The dark part or shadow part of the surface F to be cleaned may appear dark in both the infrared light image and the visible light image, while the area of the liquid object W may appear dark in the infrared light image and may appear brighter in the visible light image than in the infrared light image. Accordingly, the cleaning robot 1000 may determine an area in the difference image whose size is larger than a reference value compared to the surrounding area as the area of the liquid object W.

[0054] According to an embodiment of the present disclosure, after the infrared light image and the visible light image are input as inputs into a machine learning model, the cleaning robot 1000 may identify whether there is a liquid object W on the surface F to be cleaned and the area of the liquid object W on the surface F to be cleaned based on the area of the liquid object W output by the machine learning model.

[0055] According to an embodiment of the present disclosure, the cleaning robot 1000 can generate a synthetic image by combining an infrared light image and a visible light image, and based on the region of the liquid object W in the output synthetic image after the synthetic image is input into a machine learning model, identify whether there is a liquid object W on the surface F to be cleaned and the region of the liquid object W on the surface F to be cleaned.

[0056] According to an embodiment of the present disclosure, the cleaning robot 1000 can identify whether there is a liquid object W on the surface F to be cleaned and the region of the liquid object W on the surface F to be cleaned based on the region of the liquid object W in the infrared light image output after the infrared light image is input into the first machine learning model and the region of the liquid object W in the visible light image output after the visible light image is input into the second machine learning model.

[0057] According to an embodiment of the present disclosure, based on determining that there is a liquid object W on the surface F to be cleaned, the cleaning robot 1000 can output a notification that there is a liquid object W on the surface F to be cleaned.

[0058] According to an embodiment of the present disclosure, based on detecting a liquid object W on the travel path when moving along the travel path, the cleaning robot 1000 can change the travel path to avoid the liquid object W.

[0059] In real life, there may be a situation where a part of the surface F to be cleaned becomes wet due to pet urine, spilled drinks, etc. In this case, when the cleaning robot 1000 sucks in the liquid object W, the cleaning robot 1000 may malfunction due to the sucked-in liquid object W. In addition, when the cleaning robot 1000 continues to move forward and passes through the liquid object W, the cleaning robot 1000 may spread the dirt caused by the liquid object W.

[0060] According to an embodiment of the present disclosure, the cleaning robot 1000 can detect the presence or absence of a liquid by using a pair of electrodes to which a voltage is applied. When the liquid comes into contact with the electrodes, the resistance value between the electrodes changes, and the cleaning robot 1000 can detect the presence or absence of the liquid by comparing the changed resistance value with a predetermined threshold. However, this method involves the contact between the electrodes and the liquid. The cleaning robot 1000 needs to reach the liquid object W, and even when the cleaning robot 1000 stops or changes direction after detecting the liquid object W, the cleaning robot 1000 may not be able to avoid the liquid object W.

[0061] According to an embodiment of the present disclosure, the cleaning robot 1000 can detect a liquid only based on a visible light image. However, when detecting a liquid only based on a visible light image, the cleaning robot 1000 cannot detect a transparent liquid (e.g., pet urine or water).

[0062] According to an embodiment of the present disclosure, the cleaning robot 1000 can capture a thermal image through an infrared camera, and when a region with a predetermined temperature difference from the surrounding area is detected in the captured thermal image, the detected region is determined as the region of the liquid object W. However, when there is almost no temperature difference between the liquid object W and the surrounding area of the liquid object W, the cleaning robot 1000 cannot detect the liquid object W. For example, even if the surface F to be cleaned is wet with pet urine, it cannot be detected after a certain period of time.

[0063] By determining whether there is a liquid object W on the surface F to be cleaned based on the surface reflectivity of the liquid object W when the incident angle is greater than or equal to the threshold angle, or by detecting whether there is a liquid object W on the surface F to be cleaned based on the light absorption characteristics of water according to the light wavelength, even when the cleaning robot 1000 does not directly contact the liquid object W, even when there is almost no temperature difference between the liquid object W and its surrounding area, and even when the liquid object W is a transparent liquid, the cleaning robot 1000 can accurately identify the liquid object W with high precision.

[0064] Figure 2 A block diagram of the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0065] Referring to Figure 2 , the cleaning robot 1000 may include a processor 1100, a traveling module 1200, a light emitting unit 1800, a memory 1400, a visible light sensor 1720, an infrared light sensor 1710, and a cleaning module 1900.

[0066] The memory 1400 stores various information, data, instructions, programs, etc. required for the operation of the cleaning robot 1000. The memory 1400 may include at least one of a volatile memory, a non-volatile memory, or a combination thereof. The memory 1400 may store a map of the area to be cleaned and a traveling path.

[0067] The processor 1100 of the cleaning robot 1000 can control the overall operation of the cleaning robot 1000. The processor 1100 of the cleaning robot 1000 can execute a program stored in the memory 1400 to control the light emitting unit 1800, the traveling module 1200, the visible light sensor 1720, the infrared light sensor 1710, and the cleaning module 1900.

[0068] The cleaning robot 1000 may include at least one processor. The cleaning robot 1000 may include one processor, or may include multiple processors. At least one processor according to the present disclosure may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a multi-core integrated (MIC) processor, a digital signal processor (DSP), and a neural processing unit (NPU). At least one processor may be implemented in the form of an integrated system-on-chip (SoC) including one or more electronic components. Each of the at least one processors 1100 may be implemented as separate hardware (H / W). The at least one processor 1100 may be referred to as a MICOM (microcomputer, microprocessor computer, or microprocessor controller), a microprocessor unit (MPU), or a microcontroller unit (MCU).

[0069] At least one processor 1100 according to the present disclosure may be implemented as a single-core processor or a multi-core processor.

[0070] The light emitting unit 1800 may emit light. For example, the light emitting unit 1800 may be arranged to face a detection area on the surface to be cleaned and emit light to the detection area. The light emitting unit may include, but is not limited to, an LED or a lamp. The light source of the light emitting unit 1800 may include an infrared light source and an ultraviolet light source.

[0071] The light emitting unit 1800 may include one light source, or may include a pair of light sources 1800a and 1800b, as Figure 1 shown. When the light emitting unit 1800 includes a pair of light sources 1800a and 1800b, the pair of light sources 1800a and 1800b may be arranged at a certain interval in the horizontal direction.

[0072] According to an embodiment of the present disclosure, the light emitting unit 1800 may include an LED lamp and may generate infrared light with a peak at 970 nm. For example, the light emitting unit 1800 may generate infrared light between 960 nm and 980 nm with a peak at 970 nm.

[0073] The light emitting unit 1800 may emit radially diffused infrared light to the lower front of the cleaning robot 1000. Accordingly, the front of the path of the cleaning robot 1000 may be illuminated substantially uniformly with infrared light.

[0074] The infrared light sensor 1710 may receive infrared light reflected from a detection area on the surface to be cleaned.

[0075] According to an embodiment of the present disclosure, the infrared light sensor 1710 may include a band-pass filter for allowing only light with an absorption wavelength to pass through, so as to receive only light with an absorption wavelength.

[0076] For example, the infrared light sensor 1710 can be implemented by removing an infrared light blocking filter from a general image sensor that generates an image by converting the intensity of visible light into an electrical signal, and setting a band - pass filter that allows only light with an absorption wavelength to pass through. The general image sensor can be, for example, a charge - coupled device (CCD) sensor or a complementary metal - oxide - semiconductor (CMOS) sensor.

[0077] The directions of the infrared light sensor 1710 and the visible light sensor 1720 are towards the direction of capturing an image of a detection area on a surface to be cleaned in front of the path of the robot body. The visible light sensor 1720 and the infrared light sensor 1710 can be set to capture images of the same detection area.

[0078] According to an embodiment of the present disclosure, the light emitting unit 1800 and the infrared light sensor 1710 can be implemented as a single time - of - flight (ToF) sensor. The ToF sensor can include a light emitting unit 1800 configured to emit infrared light and an infrared light sensor 1710 configured to receive the reflected infrared light. The processor 1100 can control the light emitting unit 1800 of the ToF sensor to emit infrared light towards the surface to be cleaned, and control the infrared light sensor 1710 of the ToF sensor to detect the intensity of the infrared light that is reflected and then received.

[0079] By using the ToF sensor to obtain the distance between the ToF sensor and an object and detect the intensity of the received infrared light, even if the cleaning robot 1000 is not equipped with a separate infrared light source, the cleaning robot 1000 can use the existing ToF sensor to detect the intensity of the infrared light.

[0080] The visible light sensor 1720 can be a color image sensor configured to identify the red, green, and blue (RGB) of light, or can be a grayscale image sensor configured to identify only the brightness of light.

[0081] The visible light sensor 1720 can be a general camera (e.g., an RGB camera) configured to detect visible light to generate a color image (e.g., an RGB image). For example, the visible light sensor 1720 can be a general image sensor (e.g., a CCD sensor or a CMOS sensor) including an infrared light blocking filter. The visible light sensor 1720 can be oriented towards the direction of capturing an image of an area including the surface to be cleaned in front of the path of the robot body, and can be configured to capture an image of the traveling area in front of the robot body. In addition, the image captured by the visible light sensor 1720 can be a grayscale image or a black - and - white image instead of a color image.

[0082] The light emitting unit 1800, the infrared light sensor 1710, and the visible light sensor 1720 may be disposed in the cleaning robot 1000 to face the detection area of the surface to be cleaned.

[0083] The traveling module 1200 may move the cleaning robot 1000. The traveling module 1200 is driven according to the control signal set by the processor 1100. The processor 1100 sets the traveling path, generates a control signal for moving the cleaning robot 1000 according to the traveling path, and outputs the control signal to the traveling module 1200. The traveling module 1200 may include a motor that rotates the wheels of the cleaning robot 1000, a timing belt installed to transmit the power generated by the rear wheels to the front wheels, and the like.

[0084] The cleaning module 1900 may clean the surface to be cleaned. The cleaning module 1900 may include a dry cleaning module (not shown) configured to suck in dust and garbage and a wet cleaning module (not shown) configured to clean the surface to be cleaned with a wet mop. The dry cleaning module may include a brush, a brush motor, a dust collection container, a dust separator, a suction motor, and the like. The wet cleaning module may include a mop pad, a mop pad motor, a mop pad lifting device module, a water container, a water supply motor, and the like.

[0085] The processor 1100 may control the light emitting unit 1800 to emit infrared light to the detection area on the surface to be cleaned in front of the cleaning robot 1000.

[0086] According to an embodiment of the present disclosure, the processor 1100 may control the light emitting unit 1800 to emit light having an absorption wavelength determined based on the light absorption characteristics of water as a peak wavelength to the detection area.

[0087] In addition, the processor 1100 may receive the infrared light reflected from the detection area through the infrared light sensor 1710. In addition, the processor 1100 may receive the visible light reflected from the detection area through the visible light sensor 1720.

[0088] In addition, the processor 1100 may determine whether there is a liquid object in the detection area based on the intensity of the received infrared light and the visible light.

[0089] According to an embodiment of the present disclosure, the processor 1100 may calculate the reflectivity of the infrared light emitted to the detection area based on the intensity of the received infrared light. Based on the visible light and whether the calculated reflectivity is greater than or equal to a threshold reflectivity, the processor 1100 may determine whether there is a liquid object in the front detection area.

[0090] Based on determining that there is a liquid object in the front detection area, the processor 1100 may control the traveling module 1200 to move the cleaning robot 1000 to avoid the liquid object.

[0091] In addition, the processor 1100 may control the cleaning module 1900 to clean the surface to be cleaned.

[0092] Figure 3a The lower surface of the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0093] Referring Figure 3a , the cleaning robot 1000 may include a robot body 2 having a flat box-like appearance. A suction unit 3 may be provided on the lower surface of the front end of the robot body 2. The suction unit 3 may include a suction port 3a extending in the horizontal direction. In addition, a brush 3b that rotates about an axis between both sides of the suction port 3a may be provided inside the suction port 3a.

[0094] Although not shown, the robot body 2 may have a built-in suction motor, a dust collection container, a brush motor, etc., and when the cleaning robot 1000 is working, it may drive the brush motor to wipe off the dust on the surface to be cleaned through the brush 3b, and at the same time drive the suction motor to suck the dust into the dust collection container through the suction port 3a. According to an embodiment of the present disclosure, in addition to including the suction port 3a and the brush 3b, the cleaning robot 1000 may further include a mop module configured to perform a mopping function.

[0095] A pair of drive wheels 4a and 4b may be installed on the left and right sides of the lower surface of the robot body 2. Driven wheels 5a and 5b are installed at the upper and lower central portions of the lower surface of the robot body 2. Each of the drive wheels 4a and 4b may be configured to move forward and backward independently under the drive control of a motor in the cleaning robot 1000. The traveling module 1200 may include a pair of drive wheels 4a and 4b or driven wheels 5a and 5b, and may move the cleaning robot 1000 on the surface to be cleaned. Accordingly, the cleaning robot 1000 may move forward, backward, turn left or turn right, and may travel freely on the surface to be cleaned. The cleaning robot 1000 may move forward, which is a basic traveling operation. In addition, the cleaning robot 1000 may move backward when avoiding obstacles.

[0096] Figure 3b The front surface of the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0097] The light emitting unit 1800 may include a pair of light sources 1800a and 1800b. The light sources 1800a and 1800b may be arranged at a certain interval in the horizontal direction.

[0098] Although Figure 3b not shown in, the light emitting unit 1800 may include only one light source. In addition, the light emitting unit 1800 may include three or more light sources.

[0099] The light source (1800a or 1800b) of the light emitting unit 1800 may be set to emit infrared light obliquely downward in front of the cleaning robot 1000. Accordingly, the light source (1800a or 1800b) may emit infrared light obliquely to radially spread below the front of the cleaning robot 1000. Correspondingly, the surface to be cleaned in front of the path of the cleaning robot 1000 (including the detection area described below) may be illuminated by the infrared light substantially uniformly.

[0100] The infrared light sensor 1710 may also be oriented in a direction for capturing an image of the surface to be cleaned in front of the path of the robot body 2, and thus may capture an image of the traveling area in front of the robot body 2.

[0101] According to an embodiment of the present disclosure, in addition to detecting a liquid object, the infrared light sensor 1710 may also be driven by TOF to measure the distance between an obstacle and the robot body 2. For example, based on the time point when the infrared light is output from the light emitting unit 1800 and the time point when the output infrared light is reflected by the obstacle and then received by the infrared light sensor 1710, the cleaning robot 1000 may calculate the round-trip time for the infrared light output from the light emitting unit 1800 to be reflected by the obstacle and then return. In addition, the cleaning robot 1000 may calculate the distance between the obstacle and the robot body 2 based on the calculated round-trip time.

[0102] Correspondingly, the cleaning robot 1000 may detect whether there is a liquid object on the surface to be cleaned in front of the traveling path by using a single infrared light sensor 1710 and determine the distance between the obstacle and the robot body 2. In addition, the cleaning robot 1000 having the function of measuring the distance between the obstacle and the robot body 2 may be equipped with the function of detecting a liquid object by only updating the software program even without adding a separate infrared light sensor 1710.

[0103] The visible light sensor 1720 may be a general camera (e.g., an RGB camera) configured to detect visible light to capture a color image (e.g., an RGB image). The visible light sensor 1720 may be oriented in a direction for broadly capturing the traveling area in front of the traveling path of the robot body 2 including the surface to be cleaned (including the detection area), and may be set to capture an image of the traveling area during traveling. In addition, the image captured by the visible light sensor 1720 may be a grayscale image or a black-and-white image instead of a color image.

[0104] The visible light sensor 1720 and the infrared light sensor 1710 may be separately mounted from each other. The mounting interval between the visible light sensor 1720 and the infrared light sensor 1710 is not limited to a specific value, but the visible light sensor 1720 and the infrared light sensor 1710 may be set such that their image capture areas include the same detection area.

[0105] In addition, the visible light sensor 1720 and the infrared light sensor 1710 may be provided in the cleaning robot 1000 as an infrared-RGB (IR-RGB) camera.

[0106] Figure 4 is a flowchart of a method for avoiding a liquid object performed by the cleaning robot 1000 according to an embodiment of the present disclosure.

[0107] In operation S410, the cleaning robot 1000 may emit infrared light to a detection area on a surface to be cleaned in front of the cleaning robot.

[0108] According to an embodiment of the present disclosure, the cleaning robot 1000 may determine the presence or absence of a liquid object by using the characteristic that the surface reflectivity of a liquid object is high when the incident angle of infrared light is greater than or equal to a threshold angle. In this case, the cleaning robot 1000 may control the provided light emitting unit such that the incident angle of the light is greater than or equal to the threshold angle to emit infrared light to the detection area.

[0109] According to an embodiment of the present disclosure, the cleaning robot 1000 may determine the presence or absence of a liquid object by using the characteristic that the absorption rate of infrared light is high when infrared light having a specific wavelength is emitted to water. In this case, the cleaning robot 1000 may emit infrared light having an absorption wavelength determined based on the light absorption characteristic of water as a peak wavelength to a detection area on a surface to be cleaned in front of the cleaning robot 1000.

[0110] In operation S420, the cleaning robot 1000 may receive the infrared light reflected from the detection area through the infrared light sensor of the cleaning robot 1000.

[0111] The infrared light sensor may be installed in the cleaning robot 1000 at a position and an angle for photographing the detection area.

[0112] In operation S430, the cleaning robot 1000 may receive the visible light reflected from the detection area through the visible light sensor of the cleaning robot 1000.

[0113] The visible light sensor may also be provided in the cleaning robot 1000 at a position and an angle for photographing the detection area.

[0114] The cleaning robot 1000 may control the infrared light sensor and the visible light sensor to photograph the detection area almost simultaneously. In addition, the cleaning robot 1000 may store the received visible light data and infrared light data as a pair of data.

[0115] In operation S440, the cleaning robot 1000 can determine whether there is a liquid object in the detection area based on the intensity of the received infrared light and visible light.

[0116] The cleaning robot 1000 can generate an infrared light image representing the detection area based on the intensity of the received infrared light.

[0117] When the infrared light is emitted onto the liquid object at an incident angle greater than the threshold angle, the liquid object may appear darker in the captured infrared light image of the detection area. In addition, when the infrared light with the absorption wavelength as the peak wavelength is emitted onto the liquid object, the liquid object may also appear darker in the captured infrared light image of the detection area. Accordingly, the cleaning robot 1000 can determine the area where the liquid object is located as the area where the brightness value in the infrared light image is less than or equal to the reference brightness.

[0118] The cleaning robot 1000 can generate a visible light image representing the detection area based on the received visible light.

[0119] The cleaning robot 1000 can more accurately determine whether there is a liquid object in the detection area based on the infrared light image and the visible light image.

[0120] For example, when there is an area in the infrared light image where the brightness value is less than or equal to the first reference brightness, and the brightness value of the corresponding area in the visible light image is greater than or equal to the second reference brightness, the cleaning robot 1000 can determine that there is a liquid object in the area where the brightness value is less than or equal to the first reference brightness.

[0121] In addition, for example, the cleaning robot 1000 can input the infrared light image and the visible light image as inputs into a machine learning model, and determine whether there is a liquid object in the detection area based on the output of the machine learning model.

[0122] In operation S450, based on determining that there is a liquid object in the detection area, the cleaning robot 1000 can control the travel module to move the cleaning robot to avoid the liquid object, and control the cleaning module to clean the surface to be cleaned.

[0123] Based on detecting a liquid object in the detection area when moving along the travel path, the cleaning robot 1000 can change the direction of the robot body 2 or retreat.

[0124] In addition, when no liquid is detected in the detection area, the cleaning robot 1000 can travel along the set travel path without avoiding the detection area.

[0125] Figure 5 It is a flowchart of a method for avoiding a liquid object based on the light absorption characteristics of water performed by the cleaning robot 1000 according to an embodiment of the present disclosure.

[0126] In operation S510, the cleaning robot 1000 may emit infrared light onto a detection area on a surface to be cleaned in front of the cleaning robot 1000, and the infrared light has an absorption wavelength determined based on the light absorption characteristics of water as a peak wavelength.

[0127] The range of the absorption wavelength may be between 930 nm and 1030 nm. The absorption wavelength may refer to a wavelength having a higher light absorption rate than adjacent wavelengths according to the light absorption characteristics of water. In addition, according to an embodiment of the present disclosure, the absorption wavelength may be determined within the range of the near-infrared region.

[0128] The cleaning robot 1000 may control the light emitting unit to emit light having the absorption wavelength as a peak wavelength. According to an embodiment of the present disclosure, the cleaning robot 1000 may emit light having the absorption wavelength as a peak wavelength in the near-infrared region.

[0129] In operation S520, the cleaning robot 1000 may receive, through an infrared light sensor of the cleaning robot 1000, infrared light having the absorption wavelength among the infrared light reflected from the detection area.

[0130] The infrared light sensor may include a band-pass filter for blocking visible light and receiving light having the absorption wavelength.

[0131] In operation S530, the cleaning robot 1000 may receive visible light reflected from the detection area through a visible light sensor of the cleaning robot 1000.

[0132] According to an embodiment of the present disclosure, operations S520 and S530 may be performed almost simultaneously. For example, during travel, the cleaning robot 1000 may simultaneously control the infrared light sensor and the visible light sensor to receive infrared light through the infrared light sensor and receive visible light through the visible light sensor in the light reflected from the same area of the surface to be cleaned.

[0133] In operation S540, the cleaning robot 1000 may determine whether there is a liquid object in the detection area based on the intensity of the received infrared light and the visible light.

[0134] The cleaning robot 1000 may determine whether there is a liquid object in the area photographed by the infrared light sensor and the visible light sensor.

[0135] According to an embodiment of the present disclosure, the cleaning robot 1000 may generate an infrared light image corresponding to the intensity of the received infrared light. In addition, the cleaning robot 1000 may generate a visible light image corresponding to the intensity of the received visible light. In addition, the cleaning robot 1000 may determine whether there is a liquid object on the surface to be cleaned based on the infrared light image and the visible light image.

[0136] According to an embodiment of the present disclosure, after an infrared light image and a visible light image are input as inputs into a machine learning model, the cleaning robot 1000 can identify whether there is a liquid object on the surface to be cleaned in the captured image based on the region of the liquid object in the visible light image or the infrared light image output from the machine learning model.

[0137] According to an embodiment of the present disclosure, the cleaning robot 1000 can generate a synthetic image by combining the infrared light image and the visible light image, and after the synthetic image is input as an input into the machine learning model, identify whether there is a liquid object on the surface to be cleaned in the captured image based on the region of the liquid object in the visible light image output from the machine learning model.

[0138] According to an embodiment of the present disclosure, the cleaning robot 1000 can identify whether there is a liquid object on the surface to be cleaned in the captured image based on the region of the liquid object in the infrared light image output from the first machine learning model after the infrared light image is input as an input into the first machine learning model and the region of the liquid object in the visible light image output from the second machine learning model after the visible light image is input as an input into the second machine learning model.

[0139] According to an embodiment of the present disclosure, the cleaning robot 1000 can determine whether there is a liquid object on the surface to be cleaned in the captured image based on the difference image between the infrared light image and the visible light image.

[0140] In operation S550, based on determining that there is a liquid object in the detection area, the cleaning robot 1000 can control the traveling module to move the cleaning robot 1000 to avoid the liquid object, and control the cleaning module to clean the surface to be cleaned.

[0141] According to an embodiment of the present disclosure, based on determining that there is a liquid object on the surface to be cleaned ahead, the cleaning robot 1000 can determine the position of the cleaning robot 1000 on the map of the captured area of the surface to be cleaned based on the position of the cleaning robot 1000 on the map representing the area to be cleaned, change the traveling path to avoid the liquid object based on the position on the map of the captured area, and move the cleaning robot 1000 along the changed traveling path.

[0142] According to an embodiment of the present disclosure, based on determining that there is a liquid object on the surface to be cleaned ahead, the cleaning robot 1000 can determine the position of the liquid object on the map based on the position of the liquid object in the visible light image or the synthetic image and the position of the cleaning robot 1000 on the map, change the traveling path to avoid the liquid object based on the determined position of the liquid object on the map, and move the cleaning robot 1000 along the changed traveling path.

[0143] According to an embodiment of the present disclosure, based on determining that there is a liquid object on the surface to be cleaned ahead, the cleaning robot 1000 can determine the relative position of the liquid object with respect to the cleaning robot 1000 based on the position of the liquid object in the visible light image or the synthetic image, and move the cleaning robot 1000 based on the determined relative position to avoid the liquid object.

[0144] According to an embodiment of the present disclosure, based on determining that there is a liquid object on the surface to be cleaned ahead, the cleaning robot 1000 can output a notification that there is a liquid object on the surface to be cleaned ahead.

[0145] According to an embodiment of the present disclosure, based on determining that there is a liquid object on the surface to be cleaned ahead, the cleaning robot 1000 can send a notification that there is a liquid object on the surface to be cleaned ahead to the user device through the server.

[0146] According to an embodiment of the present disclosure, based on detecting a liquid object on the surface to be cleaned on the travel path when moving along the travel path, the cleaning robot 1000 can change the travel path to avoid the liquid object.

[0147] Based on detecting a liquid object on the surface to be cleaned on the travel path when moving along the travel path, the cleaning robot 1000 can change the direction of the robot body 2 or retreat.

[0148] Figure 6a The infrared light sensor 1710 of the cleaning robot 1000 configured to receive infrared light having an absorption wavelength according to an embodiment of the present disclosure is shown.

[0149] Referring to Figure 6a , the infrared light sensor 1710 may include a camera 1712 capable of detecting visible light and near-infrared light and a band-pass filter 1713 that blocks visible light and transmits light including infrared light having an absorption wavelength.

[0150] The infrared light sensor 1710 is a sensor for detecting a liquid object and may be configured based on the light absorption characteristics of water.

[0151] The absorption wavelength is a wavelength in the region below 1100 nm that can be detected by the Si semiconductor device in a general visible light camera and in which water has the highest light absorption. According to an embodiment of the present disclosure, the cleaning robot 1000 can use light of about 970 nm.

[0152] Here, a camera for detecting visible light usually can be equipped with an optical filter that blocks infrared light, but Figure 6a the camera 1712 in

[0153] Figure 6bShows the optical absorption spectrum of water according to an embodiment of the present disclosure.

[0154] Referring to Figure 6b the optical absorption spectrum of water shown in Figure 6b as shown in the upper part 610 of the figure above, water has high peaks at wavelengths of 1940 nm or 1450 nm. However, inexpensive Si semiconductors cannot detect light in this wavelength range due to their principle, and expensive devices (such as InGaAs) are required to detect light in this wavelength range, but it is difficult to install in household appliances in terms of cost.

[0155] On the contrary, even Si semiconductor devices used in general visible light cameras can detect light with wavelengths up to about 1100 nm in the near-infrared region.

[0156] As Figure 6b shown in the enlarged lower part 620 of the figure, although the light absorption is lower than that in the infrared region, the light absorption of water also occurs in the range between about 930 nm and about 1030 nm, and the peak is at about 970 nm.

[0157] Accordingly, according to an embodiment of the present disclosure, the cleaning robot 1000 can use a wavelength of 970 nm as the absorption wavelength. That is, a Figure 5 band-pass filter 1713 can be used to transmit infrared light with a wavelength of 970 nm, and the infrared light transmitted through the band-pass filter 1713 can be received by the camera 1712 (which is a general visible light camera). Therefore, the infrared light sensor 1710 can be implemented by combining the general visible light camera 1712 and the band-pass filter 1713 without using an expensive infrared light sensor.

[0158] Therefore, even an inexpensive cleaning robot 1000 can use the light absorption characteristics of water to detect liquid objects. In addition, the absorption wavelength is preferably 970 nm, which is the peak of absorption, but the absorption wavelength can also be selected in the range between 930 nm and 1030 nm where relatively high absorption is exhibited.

[0159] As described above, the infrared light sensor 1710 captures an image of the area in front of the traveling path of the robot body 2 including the surface to be cleaned. Accordingly, among the infrared light emitted by the light emitting unit 1800, only the infrared light with the absorption wavelength is detected, and an image based on the infrared light with the absorption wavelength is captured. Since the infrared light is absorbed by the water in the water-containing area, the area corresponding to the water-containing area in the captured image may appear relatively dark. Accordingly, the cleaning robot 1000 can detect liquid objects.

[0160] Accordingly, the cleaning robot 1000 can be inexpensively implemented by using a general camera with a sensitivity lower than the visible light range but capable of detecting up to about 1100 nm in principle as an infrared light sensor.

[0161] Figure 7 is a flowchart of a method for a cleaning robot 1000 to avoid a liquid object based on a reflectance corresponding to an incident angle according to an embodiment of the present disclosure.

[0162] In operation S710, the cleaning robot 1000 can emit infrared light onto a detection area on a surface to be cleaned in front of the cleaning robot.

[0163] The detection area is an area at a predetermined distance from the cleaning robot 1000 and is preset. In addition, the angles and positions of the light source, visible light sensor, and infrared light sensor can be determined in advance such that infrared light is emitted onto the detection area and the detection area is photographed.

[0164] According to an embodiment of the present disclosure, the cleaning robot 1000 can emit ultraviolet light instead of infrared light.

[0165] In operation S720, the cleaning robot 1000 can receive the infrared light reflected from the detection area through the infrared light sensor of the cleaning robot.

[0166] In operation S730, the cleaning robot 1000 can receive the visible light reflected from the detection area through the visible light sensor of the cleaning robot.

[0167] According to an embodiment of the present disclosure, operation S720 and operation S730 can be performed almost simultaneously. For example, the cleaning robot 1000 can control the infrared light sensor and the visible light sensor to photograph the detection area almost simultaneously. Among the light reflected from the detection area, the cleaning robot 1000 can receive infrared light through the infrared light sensor and can receive visible light through the visible light sensor.

[0168] In operation S740, the cleaning robot 1000 can calculate the reflectance of the infrared light emitted onto the detection area based on the intensity of the received infrared light.

[0169] The cleaning robot 1000 can generate an infrared light image of the detection area based on the intensity of the received infrared light.

[0170] Among the emitted infrared light, the infrared light not received back by the infrared light sensor can be regarded as the reflected infrared light. Accordingly, the cleaning robot 1000 can calculate the reflectance of the infrared light emitted toward the detection area based on the intensity of the received infrared light relative to the intensity of the emitted infrared light. For example, the cleaning robot 1000 can calculate the reflectance of the infrared light corresponding to each pixel constituting the infrared light image.

[0171] In operation S750, based on the visible light and whether the calculated reflectance is greater than or equal to a threshold reflectance, the cleaning robot 1000 can determine whether there is a liquid object in the detection area ahead.

[0172] According to an embodiment of the present disclosure, the cleaning robot 1000 can determine that there is a liquid object in an area where the reflectance in the detection area is greater than or equal to the threshold reflectance.

[0173] According to an embodiment of the present disclosure, when there is an area in the detection area where the reflectance is greater than or equal to the threshold reflectance (i.e., an area where the brightness is less than or equal to a first reference brightness), and the corresponding area in the visible light image has a brightness greater than or equal to a second reference brightness, the cleaning robot 1000 can determine that there is a liquid object in the area where the reflectance is greater than or equal to the threshold reflectance.

[0174] According to an embodiment of the present disclosure, the cleaning robot 1000 can generate an infrared light image based on the intensity of the received infrared light and generate a visible light image based on the received visible light.

[0175] The cleaning robot 1000 can input the infrared light image and the visible light image as inputs into a machine learning model and determine whether there is a liquid object in the detection area based on the output of the machine learning model.

[0176] In operation S760, based on determining that there is a liquid object in the detection area, the cleaning robot 1000 can control the traveling module to move the cleaning robot to avoid the liquid object and control the cleaning module to clean the surface to be cleaned.

[0177] Figure 8a Illustrated is a method of emitting infrared light to a liquid object in a detection area by the cleaning robot 1000 according to an embodiment of the present disclosure.

[0178] Refer to Figure 8a , the light emitting unit 1800 can be installed at a height 540 where the amount of reflected light on the surface of the liquid object W is greater than or equal to a reference ratio (e.g., 3%).

[0179] The detection area 530 of the infrared light sensor 1710 (hereinafter also referred to as the detection area) is the area to be detected by the cleaning robot 1000 and refers to the area to be detected by the infrared light sensor 1710. The detection area 530 can be determined according to, for example, the installation position of the infrared light sensor 1710, the viewing angle of the infrared light sensor 1710, the range of the surface to be cleaned to which the light emitting unit 1800 emits infrared light, etc.

[0180] The shortest detection distance 510 of the infrared light sensor 1710 is the shortest distance within the detection area 530 when viewed from the front surface of the cleaning robot 1000. The shortest detection distance 510 can be determined based on the installation position or viewing angle of the infrared light sensor 1710 and the turning radius of the cleaning robot 1000, but is not limited thereto.

[0181] The longest detection distance 520 of the infrared light sensor 1710 can be, for example, the maximum distance between the cleaning robot 1000 and the infrared light reflected from the surface to be cleaned F and received by the infrared light sensor 1710. The detection area 530 can be the area between the shortest detection distance 510 and the longest detection distance 520.

[0182] Figure 8b A method for determining the presence or absence of a liquid object based on the light reflectance performed by the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0183] Refer to Figure 8b , a part E i of the infrared light E output from the light emitting unit 1800 r is regularly reflected from the surface of the liquid object W, and the remaining part E t is transmitted into the liquid object. Here, as Figure 9a and Figure 9b shown, as the incident angle θ of the infrared light E i on the surface of the liquid object W increases, the proportion of the regular reflection (E r ) on the surface of the liquid object W increases.

[0184] In addition, the transmitted infrared light E t undergoes diffuse reflection and is then received by the infrared light sensor 1710. In this case, a part of the transmitted infrared light E t can be totally reflected.

[0185] As the incident angle θ of the infrared light E i on the surface of the liquid object W increases, the proportion of the regular reflection (E r ) on the surface of the liquid object W increases, and thus, the proportion of the infrared light transmitted into the liquid object and then received by the infrared light sensor 1710 decreases.

[0186] The relationship between the incident component (E i ) and the reflected component (E r ) of the infrared light at the boundary of the liquid object W is the following formula (1).

[0187] Formula (1):

[0188]

[0189] In formula (1), R represents the reflectance.

[0190] The reflectance R can be calculated based on the following formulas (2) to (4).

[0191] Formula (2):

[0192]

[0193] Formula (3):

[0194]

[0195] Formula (4):

[0196]

[0197] In formulas (2) to (4), θ represents the incident angle, R p represents the p-polarized reflectance, R s represents the s-polarized reflectance, n1 represents the refractive index of air (=1), and n2 represents the refractive index of the liquid.

[0198] Here, the reflectance R also varies according to polarization. For example, when using an LED light source as the light source, no polarization occurs. Based on this background, the reflectance R can be the average of the reflectances of the two polarizations.

[0199] From formulas (1) to (4) and Figure 9a and Figure 9b it can be seen that the reflectance of the liquid surface for infrared light incident at a high incident angle θ is very high. Accordingly, as the incident angle on the surface of the liquid object W increases, the amount of infrared light transmitted through the liquid object W and then received by the infrared light sensor 1710 decreases. Therefore, as the incident angle increases, in the image captured by the infrared light sensor 1710, the liquid object may appear darker than the surrounding area of the liquid object.

[0200] Accordingly, as the incident angle θ increases, the difference between the amount of light received from the liquid object W and the amount of light received from objects other than the liquid object W increases. Therefore, the liquid object W can be detected more accurately. Therefore, in order to minimize the influence of ambient light, the height of the light source from the surface to be cleaned is preferably low.

[0201] Figure 9a and Figure 9b shows the relationship between the infrared light incident angle and the regular reflection ratio according to an embodiment of the present disclosure.

[0202] Figure 9a The curve graph of

[0203] Referring to Figure 9a the upper figure above, it can be seen that as the incident angle increases, the reflection coefficient R (i.e., the proportion of regular reflection) increases.

[0204] Figure 9a The lower figure below is an enlarged view showing the reflection coefficient R when the incident angle in the upper figure is from 40° to 60°. It can be seen that when the incident angle is 48° or greater, the reflection coefficient R is 3% or greater.

[0205] Figure 9b The curve diagram below shows the proportion of regular reflection obtained according to the incident angle when infrared light with a wavelength of 850 nm is emitted onto the oil. Similar to water, it can be seen that as the incident angle increases, the proportion of regular reflection increases.

[0206] The wavelength of the infrared light emitted by the light emitting unit 1800 is not limited to 850 nm, and the same effect can be obtained even when using infrared light of other wavelengths. In addition, ultraviolet light can be used instead of infrared light as the light source of the light emitting unit 1800. However, from the perspective of manufacturing cost, infrared light may be more preferable.

[0207] The liquid object may include organic compounds such as alcohol in addition to water or oil.

[0208] Generally speaking, the reflectivity of a liquid is determined by the refractive index of the liquid, and the lower the refractive index, the lower the reflectivity. Since the refractive index of organic compounds such as alcohol is greater than that of water, the cleaning robot 1000 can also detect liquids such as organic compounds.

[0209] In addition, experiments have confirmed that even when solid substances such as milk or juice are mixed into the water, the ratio of the incident angle to regular reflection is not significantly different from the characteristics of water.

[0210] In addition, experiments have confirmed that when the amount of light reflected on the surface of the liquid object by the emitted light is at least 3%, the difference between the amount of light received from the liquid object W and the amount of light received from an object other than the liquid object W increases. Therefore, the liquid object W and the general surface to be cleaned can be accurately distinguished from each other.

[0211] Therefore, the cleaning robot 1000 includes a light source located at a position where the amount of light reflected on the surface of the liquid object by the light emitted in the cleaning robot 1000 is at least 3%. Therefore, various liquids present on the floor at home can be detected.

[0212] In the case where only one light source is provided, the light source can be provided in the cleaning robot 1000 at a height such that the amount of reflected light on the surface of the liquid object is at least 3%. In addition, when two or more light sources are provided, at least one of the light sources can be provided in the cleaning robot 1000 at a height such that the amount of reflected light on the surface of the liquid object is at least 3%. However, in the case where a plurality of light sources are installed at intervals of each other in the horizontal direction in order to expand the field of view of the cleaning robot 1000, it is preferable that the heights of the light sources are equal to each other.

[0213] Figure 10a A method of determining whether there is a liquid object W on a surface to be cleaned by the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0214] Referring to Figure 10a , the cleaning robot 1000 can use a machine learning model to determine the presence or absence of the liquid object W.

[0215] Before determining the presence or absence of the liquid object based on the infrared light image 710 and the visible light image 720, the cleaning robot 1000 can determine whether the two images represent the same region at the same time point. The cleaning robot 1000 can match the infrared light image 710 with the visible light image 720 such that the infrared light image 710 and the visible light image 720 represent the same region at the same time point.

[0216] In the infrared light image 710, the liquid object W appears darker than the surrounding area of the liquid object W. In addition, when the liquid object W is transparent, it may be difficult to identify the liquid object W in the visible light image 720.

[0217] According to an embodiment of the present disclosure, the cleaning robot 1000 can generate a synthetic image by combining the infrared light image 710 with the visible light image 720, and obtain information about the liquid object from the machine learning model 23 by inputting (applying) the generated synthetic image to the machine learning model 23 trained for the synthetic image.

[0218] An image emphasizing the leading edge of the liquid can be obtained by direct reflection of the light irradiated to the liquid edge or by total reflection in the liquid ( Figure 8b total reflection). In this case, by combining the infrared light image 710 obtained from the infrared light sensor 1710 with the visible light image 720 obtained from the visible light sensor 1720, the influence of the shape, color, and shadow of the liquid can be suppressed, and thus the detection accuracy of the liquid can be improved.

[0219] According to an embodiment of the present disclosure, the cleaning robot 1000 does not combine the infrared light image 710 with the visible light image 720, but applies the infrared light image 710 and the visible light image 720 to the fully trained machine learning model 23. For example, the positions of the corresponding images can be adjusted to change the images into data of height × width × 4 channels (the 4 channels include the infrared light image, the R image, the G image, and the B image), and the changed 4-channel data can be input into the fully trained machine learning model 23 for 4-channel data.

[0220] According to an embodiment of the present disclosure, the cleaning robot 1000 can determine the presence or absence of a liquid object based only on the infrared light image 710. In this case, the cleaning robot 1000 includes the fully trained machine learning model 23 corresponding to the infrared light image 710, and can apply the infrared light image 710 to the machine learning model 23. Specifically, the cleaning robot 1000 can input data of height × width × 1 channel (infrared light image) into the fully trained machine learning model 23.

[0221] According to an embodiment of the present disclosure, the cleaning robot 1000 can determine the presence or absence of a liquid object based on a plurality of infrared light images captured at consecutive time points and a plurality of visible light images captured at consecutive time points.

[0222] When the output of the machine learning model 23 indicates the position of the region 750 of the liquid object W in the detection region image 740 of the surface F to be cleaned, and the region 750 of the liquid object W is larger than a predetermined size, the cleaning robot 1000 can determine that there is a liquid object W in the captured detection region.

[0223] When the output of the machine learning model 23 indicates the position of the liquid object W in the detection region image 740 of the surface F to be cleaned, the cleaning robot 1000 can determine the position of the liquid object W on the cleaning area map based on the current position of the cleaning robot 1000 on the cleaning area map and the position of the liquid object W in the detection region image 740. In addition, the cleaning robot 1000 can change (modify) the travel path based on the position of the liquid object W on the map.

[0224] According to an embodiment of the present disclosure, when the output of the machine learning model 23 indicates that no liquid object W is detected in the detection region image 740, the cleaning robot 1000 can determine that there is no liquid object W on the surface to be cleaned.

[0225] Figure 10b A method for determining whether there is a liquid object W on the surface to be cleaned by the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0226] Refer to Figure 10b, the cleaning robot 1000 can use a machine learning model to determine the presence or absence of the liquid object W.

[0227] According to an embodiment of the present disclosure, the cleaning robot 1000 can generate a synthetic image 730 by combining the infrared light image 710 and the visible light image 720. In addition, the cleaning robot 1000 can input the generated synthetic image 730 as an input into a preset and fully trained machine learning model to determine the presence or absence of the liquid object W. The cleaning robot 1000 can determine the presence or absence of the liquid object W on the surface to be cleaned and the area of the liquid object W on the surface to be cleaned based on the output of the machine learning model.

[0228] On Figure 10b An example of the infrared light image 710 and the visible light image 720 is shown on the left side. The infrared light image 710 and the visible light image 720 can be images including the same area of the surface to be cleaned. The lower parts 712 and 722 of the infrared light image 710 and the visible light image 720 are outside the imaging field of view, and the upper parts 711 and 721 are within the imaging field of view. In the infrared light image 710, the liquid object W looks darker than the surrounding area of the liquid object W. In addition, when the liquid object W is transparent, it may be difficult to identify the liquid object W in the visible light image. The cleaning robot 1000 can generate a synthetic image 730 by combining the infrared light image 710 and the visible light image 720, and obtain information about the liquid object from the machine learning model 23 by inputting (applying) the generated synthetic image 730 into the machine learning model 23. For example, the cleaning robot 1000 can generate a synthetic image 730 by converting the infrared light image 710 to grayscale and replacing the G channel and the B channel in the RGB channels of the visible light image 720 with the grayscale infrared light image.

[0229] In addition, for example, the cleaning robot 1000 can obtain a three-channel synthetic image combining the infrared light image and the visible light image by converting the infrared light image 710 to grayscale and replacing the image of the R channel in the RGB channels of the visible light image 720 with the grayscale infrared light image.

[0230] In addition, for example, the cleaning robot 1000 can generate a synthetic image by converting the infrared light image 710 to grayscale and replacing the G or B channel in the RGB channels of the visible light image 720 with the grayscale infrared light image.

[0231] In addition, for example, the cleaning robot 1000 can generate a composite image as a 4-channel image by using the infrared light image 710 as the transmittance. The method of combining the infrared light image 710 and the visible light image 720 is not limited to the above description, but can be any method as long as it is a composite method for obtaining information about both the infrared light image 710 and the visible light image 720. The machine learning model of the cleaning robot 1000 can be trained by using training data before driving the cleaning robot 1000. The machine learning model can also be referred to as an artificial intelligence (AI) model or a neural network model.

[0232] The neural network model can include multiple neural network layers (e.g., an input layer, an intermediate layer (hidden layer), and an output layer). Each layer in the neural network layer has multiple weight values and performs neural network arithmetic operations through arithmetic operations between the arithmetic operation results of the previous layer and the multiple weight values. The multiple weight values in each layer of the neural network layer can be optimized through the results of training the neural network model. For example, the multiple weight values can be updated to reduce or minimize the loss or cost value of the neural network model during the training process. Examples of neural networks include, but are not limited to, deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent DNNs (BRDNNs), or deep Q networks.

[0233] The neural network model according to an embodiment of the present disclosure can be a model for inferring the position of a liquid object. Inference / prediction is a technology for determining information for logical reasoning and prediction, and includes knowledge / probability-based reasoning, optimization prediction, preference-based planning, recommendation, etc.

[0234] According to the present disclosure, functions related to AI are executed by at least one processor and a memory. The at least one processor can include one or more processors. In this case, the at least one processor can be a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), or a digital signal processor (DSP); a dedicated graphics processor, such as a graphics processing unit (GPU) or a vision processing unit (VPU); or a dedicated artificial intelligence processor, such as a neural processing unit (NPU). The at least one processor performs control such that input data is processed according to predefined operation rules or an AI model (e.g., a neural network model) stored in the memory. Alternatively, in the case where the at least one processor is a dedicated AI processor, the dedicated AI processor can be designed to have a hardware structure dedicated to processing a specific AI model.

[0235] Predefined operation rules or AI models (e.g., neural network models) can be generated through a training process. Here, generating through a training process can mean generating predefined operation rules or AI models that are set to perform desired characteristics (or purposes) by training a basic AI model using a learning algorithm that utilizes a large amount of training data. The training process can be executed by the device itself that performs artificial intelligence according to the present disclosure (e.g., the cleaning robot 1000), or by a separate server and / or system. Examples of learning algorithms can include, for example, supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning, but are not limited thereto.

[0236] According to an embodiment of the present disclosure, the training data can be a synthetic image of a pair of infrared light images and visible light images obtained by photographing a liquid object (e.g., water or juice) spilled on a surface to be cleaned. In this case, the position of the liquid object can be marked within the synthetic image corresponding to the pair of images. In addition, the training data can include a pair of infrared light images and visible light images of the surface to be cleaned where there is no liquid object. In this case, information indicating the absence of a liquid object can be marked as corresponding to the pair of images. In addition, the training data can include many pairs of infrared light images and visible light images for various liquid objects present on various surfaces to be cleaned.

[0237] According to an embodiment of the present disclosure, the machine learning model can be trained to output the position of the liquid object marked in the synthetic image (to infer the position of the liquid object) after inputting one synthetic image into the machine learning model. In addition, the machine learning model can be trained to output information indicating the absence of a liquid object when a synthetic image without a liquid object is input into the machine learning model.

[0238] The cleaning robot 1000 can determine the presence or absence of the liquid object W and the position of the liquid object W by using the fully trained machine learning model 23. For example, when the cleaning robot 1000 is traveling, the cleaning robot 1000 can control the infrared light sensor 1710 and the visible light sensor 1720 to photograph the surface to be cleaned in front of the traveling path. In addition, the cleaning robot 1000 can generate a synthetic image 730 by processing the infrared light image and the visible light image respectively obtained from the infrared light sensor 1710 and the visible light sensor 1720. The cleaning robot 1000 determines the presence or absence of the liquid object W and the position of the liquid object W by applying the generated synthetic image 730 to the fully trained machine learning model 23 installed thereon.

[0239] In Figure 10bOn the right side, an example of a determination result image (detection area image 740) representing the determination result of the fully trained machine learning model 23 is shown. When the output of the machine learning model 23 represents the position of the liquid object W in the composite image 730, the cleaning robot 1000 can generate the determination result image 740, in which the position of the liquid object W on the composite image 730 is marked with an edge 63.

[0240] According to an embodiment of the present disclosure, when the output of the machine learning model 23 represents the position of the liquid object W in the composite image 730 and the area of the liquid object W is greater than or equal to a predetermined size, the cleaning robot 1000 can determine that there is a liquid object W on the surface to be cleaned in the captured image.

[0241] According to an embodiment of the present disclosure, when the output of the machine learning model 23 represents the position of the liquid object W in the composite image 730, the cleaning robot 1000 can determine the position of the liquid object W on the cleaning area map based on the current position of the cleaning robot 1000 on the cleaning area map and the position of the liquid object W in the composite image 730. In addition, the cleaning robot 1000 can change (modify) the travel path based on the position of the liquid object W on the map.

[0242] According to an embodiment of the present disclosure, when the output of the machine learning model 23 indicates that no liquid object W is detected in the composite image 730, the cleaning robot 1000 can determine that there is no liquid object W on the surface to be cleaned.

[0243] Figures 11a to 11c A diagram for describing a method for determining a liquid object on a surface to be cleaned performed by the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0244] Refer to Figures 11a to 11c , the cleaning robot 1000 can generate composite images 730a, 730b, and 730c by combining infrared light images 710a, 710b, and 710c obtained by converting to grayscale and visible light images 720a, 720b, and 720c, and obtain information about the liquid object output from the machine learning model by inputting the generated composite images 730a, 730b, and 730c into the machine learning model. In addition, the cleaning robot 1000 can generate determination result images 740a, 740b, and 740c, in which the frame images 63 indicating the position of the liquid object are indicated in the composite images 730a, 730b, and 730c.

[0245] Refer to Figure 11b , even when the surface to be cleaned is a wooden floor, the cleaning robot 1000 can accurately detect the position of the liquid spilled on the wooden floor.

[0246] Reference Figure 11c , even when the surface to be cleaned is a carpet, the cleaning robot 1000 can accurately detect the position of the liquid on the carpet.

[0247] Figure 11d shows a method for determining whether there is a liquid object on the surface to be cleaned, which is performed by the cleaning robot 1000 according to an embodiment of the present disclosure.

[0248] Reference Figure 11d , even when the image of the surface to be cleaned captured is uneven (for example, due to the presence of steps or obstacles on the surface to be cleaned), the cleaning robot 1000 can determine the liquid object with high precision.

[0249] Figure 11d The upper image of Figure 11d is the composite image 730d of the surface to be cleaned with the step D. Although it is difficult to identify Figure 11d the liquid object in the composite image 730d of

[0250] with the naked eye, there is a liquid object above the step D.

[0251] Figure 12 is a flowchart of a method for detecting and avoiding a liquid object during travel, which is performed by the cleaning robot 1000 according to an embodiment of the present disclosure.

[0252] In operation S1210, when the cleaning robot 1000 starts operating and travels, the cleaning robot 1000 can receive an infrared light signal and a visible light signal through an infrared light sensor and a visible light sensor.

[0253] In operation S1220, the cleaning robot 1000 can determine whether the operation of the cleaning robot 1000 has terminated.

[0254] When it is determined in operation S1220 that the operation of the cleaning robot 1000 has not terminated, the cleaning robot 1000 can obtain an infrared light image in operation S1230 and obtain a visible light image in operation S1240.

[0255] ​​​In operation S1250, the cleaning robot 1000 can generate a synthetic image by combining an infrared light image and a visible light image.

[0256] In operation S1260, the cleaning robot 1000 can apply the generated synthetic image to the fully trained machine learning model 23.

[0257] In operation S1270, the cleaning robot 1000 can determine whether there is a liquid object on the surface to be cleaned.

[0258] When it is determined in operation S1270 that there is no liquid object on the surface to be cleaned, the cleaning robot 1000 can move forward along the travel path in operation S1280. For example, the cleaning robot 1000 can output a normal control signal to the travel module to control the robot body to move forward and travel as it is.

[0259] When it is determined in operation S1270 that there is a liquid object on the surface to be cleaned, the cleaning robot 1000 can avoid the liquid object while traveling in operation S1290. For example, the cleaning robot 1000 can output a control signal for changing the travel to the travel module and control the robot body to travel while avoiding the area of the liquid object. The cleaning robot 1000 can perform operations S1210 to S1290 until the operation of the cleaning robot 1000 terminates. Accordingly, even if there is a liquid object in the travel path, the robot body can travel while avoiding the liquid object.

[0260] Figure 13 A method for determining the presence or absence of a liquid object by the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0261] Refer to Figure 13 , the cleaning robot 1000 determines the presence or absence of a liquid object in each of the infrared light image 710 and the visible light image 720, and comprehensively determines the presence or absence of a liquid object based on each determination result. The cleaning robot 1000 finally determines the presence or absence of a liquid object by combining the determinations of the infrared light image 710 and the visible light image 720, improving the accuracy of the determination.

[0262] Specifically, the cleaning robot 1000 can apply the infrared light image 710 to the first fully trained machine learning model 23a corresponding to the infrared light image, and based on the output of the first machine learning model 23a, perform a first determination process for determining the presence or absence of a liquid object on the surface to be cleaned.

[0263] In addition, the cleaning robot 1000 may apply the visible light image 720 to a second machine learning model 23b that is fully trained and corresponds to the visible light image, and based on the output of the second machine learning model 23b, perform a second determination process for determining the presence or absence of a liquid object on the surface to be cleaned. In addition, the cleaning robot 1000 may comprehensively determine the presence or absence of a liquid object on the surface to be cleaned based on the first determination process and the second determination process.

[0264] According to an embodiment of the present disclosure, by using existing fully trained machine learning models, a fully trained first machine learning model 23a and a second machine learning model 23b that respectively correspond to the infrared light image 710 and the visible light image 720 may be constructed. In addition, the fully trained machine learning models 23a and 23b may be stored in the memory of the cleaning robot 1000.

[0265] In addition, when the cleaning robot 1000 is traveling, it may control the infrared light sensor and the visible light sensor to capture the surface to be cleaned in front of the traveling path. The cleaning robot 1000 may obtain the infrared light image 710 and the visible light image 720 from the infrared light sensor and the visible light sensor.

[0266] The cleaning robot 1000 may determine the presence or absence of a liquid object based on the obtained infrared light image 710 (first determination), and determine the presence or absence of a liquid object based on the obtained visible light image 720 (second determination). When the first determination and the second determination match, the cleaning robot 1000 may finally determine the presence or absence of a liquid object based on the matching determination.

[0267] On the contrary, when the first determination and the second determination are different from each other, the cleaning robot 1000 may evaluate the accuracy of the first determination and the second determination.

[0268] For example, in addition to outputting information on the presence or absence of a liquid object, the first machine learning model and the second machine learning model may also output the accuracy of the information. Therefore, the cleaning robot 1000 may select which determination to adopt from the first determination and the second determination by comparing the accuracy output by the first machine learning model 23a and the second machine learning model 23b with a predetermined reference value or by comparing the output accuracies with each other.

[0269] For example, when the first determination is that there is a liquid object, the second determination is that there is no liquid object, and the accuracy of the second determination is lower than the accuracy of the first determination, the cleaning robot 1000 may determine that there is a liquid object based on the first determination.

[0270] Figure 14Disclosed is a method for determining the presence or absence of a liquid object by a cleaning robot 1000 according to an embodiment of the present disclosure, based on the difference between an infrared light image and a visible light image.

[0271] Specifically, the cleaning robot 1000 can determine the presence or absence of a liquid object by calculating the difference between the infrared light image and the visible light image and comparing the calculated difference with a predetermined threshold.

[0272] Referring Figure 14 , when the cleaning robot 1000 is traveling, it can control the infrared light sensor and the visible light sensor to capture the surface to be cleaned in front of the traveling path. The cleaning robot 1000 can receive the infrared light image 710 and the visible light image 720 from the infrared light sensor and the visible light sensor.

[0273] The cleaning robot 1000 can calculate the difference between the infrared light image 710 and the visible light image 720. The cleaning robot 1000 can compare the calculated difference with a predetermined threshold. When the calculated difference is greater than or equal to the threshold, the cleaning robot 1000 can determine that there is a liquid object on the captured surface to be cleaned, and when the calculated difference is less than the threshold, the cleaning robot 1000 can determine that there is no liquid object on the captured surface to be cleaned.

[0274] According to an embodiment of the present disclosure, due to uneven illumination in practical applications, areas farther away from the cleaning robot 1000 appear darker in the infrared light image, and the influence of the color of the surface to be cleaned in the visible light image needs to be considered. Therefore, the cleaning robot 1000 can perform brightness correction according to the distance or color before calculating the difference between the infrared light image and the visible light image.

[0275] Liquid objects appear darker in the infrared light image with absorption wavelengths. Therefore, when the area of the surface to be cleaned that appears brighter in the visible light image is a liquid object, the difference between the infrared light image and the visible light image may be large. Therefore, when the area that appears brighter in the visible light image is the area of the liquid object on the surface to be cleaned, the cleaning robot 1000 can accurately determine the presence or absence of the liquid object and the area of the liquid object.

[0276] On the contrary, black areas or shadow areas on the surface to be cleaned also appear darker in the infrared light image with absorption wavelengths, even if there is no liquid object in that area. Therefore, by only using the infrared light image with absorption wavelengths, it is impossible to identify whether the dark area in the infrared light image is a liquid object area, a black part, or a shadow part. Therefore, there is a possibility of false detection of liquid objects.

[0277] In addition, black or shadow areas on the surface to be cleaned also appear darker in visible light images. Therefore, when the area on the surface to be cleaned is a black or shadow area of a non-liquid object, the difference between the infrared light image and the visible light image is small.

[0278] When the black or shadow area on the surface to be cleaned is a liquid object area, it is very likely to determine that the black or shadow area is not a liquid object area, but the case where only the black or shadow area is a liquid object area is rare. Therefore, the probability of misdetecting a liquid object area is low.

[0279] According to an embodiment of the present disclosure, the cleaning robot 1000 can determine the presence or absence of a liquid object and the area of the liquid object based only on the infrared light image 710. For example, the cleaning robot 1000 can include a machine learning model 23 that is fully trained to output the presence or absence of a liquid object and the area of the liquid object when an infrared light image is input; and can determine the presence or absence of a liquid object and the area of the liquid object by applying the captured infrared light image to the fully trained machine learning model 23.

[0280] Figure 15 A method for notifying the presence of a liquid object in an area to be cleaned performed by the cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0281] Refer to Figure 15 , when a liquid object W is detected ahead, the cleaning robot 1000 can output a notification that the liquid object W has been detected.

[0282] The cleaning robot 1000 can control the infrared light sensor 1710 and the visible light sensor 1720 to obtain an infrared light image and a visible light image of the surface to be cleaned ahead. The cleaning robot 1000 can determine whether there is a liquid object W on the captured surface to be cleaned based on the obtained infrared light image and visible light image.

[0283] According to an embodiment of the present disclosure, when there is a liquid object W on the surface to be cleaned, the cleaning robot 1000 can output a voice or a notification sound to notify the presence of the liquid object W ahead, and then avoid the liquid object W and move forward.

[0284] In addition, according to an embodiment of the present disclosure, when there is a liquid object W on the surface to be cleaned, the cleaning robot 1000 can output a voice or a notification sound to notify the presence of the liquid object W ahead, then stop for a predetermined period so that the user knows the position of the liquid object W, and then avoid the liquid object W and move forward again.

[0285] In addition, according to an embodiment of the present disclosure, based on the size of the liquid object W, the cleaning robot 1000 may determine whether to travel around the liquid object W or to travel along a set travel path while sucking in the liquid object W. When the liquid object W is small, even if the liquid object W is sucked in, there is almost no risk of the cleaning robot 1000 being contaminated or the liquid object W spreading. Therefore, only when the size of the liquid object W is greater than or equal to a predetermined reference size, the cleaning robot 1000 may travel while avoiding the liquid object W.

[0286] Figure 16 FIG. shows a method of notifying the presence of a liquid object in a to-be-cleaned area by the cleaning robot 1000 according to an embodiment of the present disclosure.

[0287] Referring to Figure 16 , the cleaning robot 1000 may output a notification of the presence of the liquid object W in the to-be-cleaned area through the user device 2000.

[0288] Based on determining the presence of the liquid object W ahead, the cleaning robot 1000 may determine the position of the liquid object W on the map of the to-be-cleaned area stored in the cleaning robot 1000. In addition, the cleaning robot 1000 may send the determined position of the liquid object W on the map to the server 3000. In this case, the cleaning robot 1000 may send the identification information of the cleaning robot 1000 and the position of the liquid object W on the map to the server 3000 based on the previously stored address information of the server 3000.

[0289] The server 3000 may send the position of the liquid object W on the map received from the cleaning robot 1000 to the user device 2000. In this case, the server 3000 may obtain the identification information of the user device 2000 corresponding to the identification information of the cleaning robot 1000, and based on the obtained identification information of the user device 2000, send the position of the liquid object W on the map to the user device 2000.

[0290] According to an embodiment of the present disclosure, when the server 3000 receives the position of the liquid object W on the map from the cleaning robot 1000, the server 3000 may immediately send a push notification of the presence of the liquid object W in the to-be-cleaned area to the user device 2000.

[0291] In response to receiving a push notification from the server 3000, the user device 2000 may display the received push notification. Based on receiving a user input for selecting the displayed push notification, the user device 2000 may display a map 150 of the area to be cleaned and an image 153 indicating the position of the liquid object W on the map 150 of the area to be cleaned. In addition, the user device 2000 may display a phrase 154 indicating the presence of the liquid object W in the area to be cleaned, or a phrase indicating that the liquid object W needs to be processed. In addition, the user device 2000 may display an image of the captured liquid object W. Accordingly, once the cleaning robot 1000 detects the liquid object W while traveling, the user can identify the presence of the liquid object W in the area to be cleaned through the user device 2000 and check the position of the liquid object W.

[0292] According to an embodiment of the present disclosure, based on receiving a user input for selecting a menu for viewing cleaning information of the cleaning robot 1000, the user device 2000 may display a map 150 of the area to be cleaned and an image 153 indicating the position of the liquid object W on the map 150 of the area to be cleaned.

[0293] According to an embodiment of the present disclosure, the user device 2000 may display an image 153 indicating the position of the liquid object W and an image indicating the travel path of the cleaning robot 1000 on the map 150 of the area to be cleaned.

[0294] Figure 17 A block diagram of a cleaning robot 1000 according to an embodiment of the present disclosure is shown.

[0295] Referring to Figure 17 , the cleaning robot 1000 may include a processor 1100, a light emitting unit 1800, a communication module 1300, a memory 1400, an input interface 1500, an output module 1600, a sensor 1700, a travel module 1200, and a cleaning module 1900. Components identical to those shown in Figure 2 are denoted by the same reference numerals.

[0296] Figure 17 All components shown in Figure 17 are not essential components of the cleaning robot 1000. The cleaning robot 1000 may be implemented with more or fewer components than those shown in

[0297] The output module 1600 may include an audio output module 1620 and a display 1610.

[0298] The audio output module 1620 may output an audio signal to the outside of the cleaning robot 1000. The audio output module 1620 may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as reproducing multimedia or recording.

[0299] The display 1610 can output image data processed by an image processor (not shown) through a display panel (not shown) under the control of the processor 1100. The display panel (not shown) can include at least one of a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, a three-dimensional (3D) display, and an electrophoretic display.

[0300] The input interface 1500 can receive user input for controlling the cleaning robot 1000. The input interface 1500 receives the user input and sends it to the processor 1100.

[0301] The input interface 1500 can include a user input electronic device, which includes a touch panel for detecting user touches, buttons for receiving user pressing operations, a roller for receiving user rotation operations, a keyboard, a dome switch, etc., but is not limited thereto.

[0302] In addition, the input interface 1500 can include a voice recognition device for voice recognition. For example, the voice recognition device can be a microphone and can receive a user's voice command or voice request. Accordingly, the processor 1100 can control the execution of an operation corresponding to the voice command or voice request.

[0303] The memory 1400 stores various information, data, instructions, programs, etc. required for the operation of the cleaning robot 1000. The memory 1400 can include at least one of a volatile memory, a non-volatile memory, or a combination thereof. The memory 1400 can include at least one of a flash memory type storage medium, a hard disk type storage medium, a multimedia card micro storage medium, a card type memory (e.g., SD or XD memory), a random access memory (RAM), a static RAM (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the cleaning robot 1000 can operate a network storage or a cloud server that performs a storage function on the Internet.

[0304] The communication module 1300 can send information to and receive information from an external device or an external server according to a protocol under the control of the processor 1100. The communication module 1300 can include at least one communication module and at least one port, and the communication module is configured to send data to and receive data from an external device (not shown).

[0305] In addition, the communication module 1300 may communicate with external devices through at least one wired or wireless communication network. The communication module 1300 may include at least one of a short-range communication module 1310, a long-range communication module 1320, or a combination thereof. The communication module 1300 may include at least one antenna for wireless communication with other devices.

[0306] The short-range communication module 1310 may include at least one communication module (not shown) configured to perform communication according to communication standards such as Bluetooth, Wi-Fi, Bluetooth Low Energy (BLE), Near Field Communication (NFC) / Radio Frequency Identification (RFID), Wi-Fi Direct, Ultra-Wideband (UWB), or Zigbee. In addition, the long-range communication module 1320 may include a communication module (not shown) configured to perform communication through an Internet communication network. In addition, the long-range communication module 1320 may include a mobile communication module configured to perform communication according to communication standards such as the third generation (3G), fourth generation (4G), fifth generation (5G), and / or sixth generation (6G).

[0307] In addition, the communication module 1300 may include a communication module capable of receiving control commands from a remote controller (not shown) at close range, such as an IR communication module.

[0308] The cleaning module 1900 may include a dry cleaning module 1910 and a wet cleaning module 1920. The dry cleaning module 1910 may include a brush, a dust collection container, a dust separator, a suction motor, etc. The wet cleaning module 1920 may include a mop pad, a mop pad motor, a mop pad lifting device module, a water container, a water supply motor, etc.

[0309] The suction motor (or vacuum motor) may suck air through the suction port of the cleaning robot 1000 by rotating a fan connected to the suction motor.

[0310] The brush may be a brush with multiple bristles or a lint brush with lint, but is not limited thereto. The brush may rotate by the driving force transmitted by the brush motor. The brush may sweep dust or foreign objects adhering to the floor and move them to the suction port of the cleaning robot 1000.

[0311] The dust sucked through the suction port of the cleaning robot 1000 and filtered by the dust separator may be stored in the dust collection container.

[0312] The mop pad motor may perform wet mopping by moving the mop pad attached to the cleaning robot 1000. The mop pad lifting device module may perform a pad lifting operation that changes the mop pad in close contact with the floor to be in close contact with the cleaning robot 1000, or a pad lowering operation that changes the mop pad in close contact with the cleaning robot 1000 to be in close contact with the floor.

[0313] The sensor 1700 may include various types of sensors.

[0314] For example, the sensor 1700 may include an infrared light sensor 1710 and a visible light sensor 1720. According to an embodiment of the present disclosure, the sensor 1700 may include an obstacle sensor, an object sensor, and an anti-fall sensor.

[0315] The obstacle sensor may output infrared or ultrasonic waves and receive the reflected signal reflected from the obstacle. The processor 1100 may control the obstacle sensor to detect whether there is an obstacle in front.

[0316] The object sensor may include a two-dimensional camera sensor and a 3D camera sensor. In addition, the visible light sensor 1720 may be used as the object sensor. The object sensor may capture an image in front of the cleaning robot 1000 and identify the type and position of the object in the captured image.

[0317] The anti-fall sensor may include an infrared light emitting unit and an infrared light receiving unit both facing the floor. The processor 1100 may control the infrared light emitting unit of the anti-fall sensor to output infrared rays to the floor and control the infrared light receiving unit to receive the reflected signal reflected from the floor. The processor may detect the distance between the cleaning robot 1000 and the floor based on the received reflected signal. In addition, the processor 1100 may identify the possibility of falling and the threshold based on the distance from the floor.

[0318] In addition, the sensor 1700 may include a plurality of sensors configured to detect information about the environment around the cleaning robot 1000. For example, the sensor 1700 may include an ultrasonic sensor, a motion sensor, etc., but is not limited thereto.

[0319] At least one processor 1100 may control the light emitting unit 1800 to emit light toward the surface to be cleaned along the traveling direction of the cleaning robot 1000, and the light has an absorption wavelength determined based on the absorption characteristics of water as the peak wavelength.

[0320] At least one processor 1100 may receive infrared light having the absorption wavelength in the light reflected from the surface to be cleaned through the infrared light sensor 1710.

[0321] At least one processor 1100 may receive visible light reflected from the surface to be cleaned through the visible light sensor 1720.

[0322] At least one processor 1100 may determine whether there is a liquid object on the surface to be cleaned based on the received infrared light and visible light.

[0323] Based on determining the presence of a liquid object on the surface to be cleaned, at least one processor 1100 may control the travel module 1200 to move the cleaning robot 1000 to avoid the liquid object.

[0324] The range of the absorption wavelength may be between 930 nm and 1030 nm.

[0325] The infrared light sensor 1710 may include a band - pass filter for blocking visible light and receiving light with the absorption wavelength.

[0326] At least one processor 1100 may generate an infrared light image of the surface to be cleaned based on the intensity signal of the received infrared light obtained from the infrared light sensor 1710.

[0327] At least one processor 1100 may generate a visible light image of the surface to be cleaned based on the intensity signal of the received visible light obtained from the visible light sensor 1720.

[0328] At least one processor 1100 may determine whether there is a liquid object on the surface to be cleaned based on the infrared light image and the visible light image.

[0329] At least one processor 1100 may generate a synthetic image by combining the infrared light image and the visible light image.

[0330] At least one processor 1100 may determine whether there is a liquid object on the surface to be cleaned based on the region of the liquid object in the visible light image output from the machine - learning model after inputting the synthetic image into the machine - learning model.

[0331] At least one processor 1100 may identify whether there is a liquid object on the surface to be cleaned based on the region of the liquid object in the infrared light image output from the first machine - learning model after inputting the infrared light image into the first machine - learning model and the region of the liquid object in the visible light image output from the second machine - learning model after inputting the visible light image into the second machine - learning model.

[0332] At least one processor 1100 may determine whether there is a liquid object on the surface to be cleaned based on the difference image between the infrared light image and the visible light image.

[0333] Based on determining the presence of a liquid object on the surface to be cleaned, at least one processor 1100 may output a notification of the presence of a liquid object on the surface to be cleaned.

[0334] Based on determining the presence of a liquid object on the surface to be cleaned, at least one processor 1100 may send a notification of the presence of a liquid object on the surface to be cleaned to the user device through the server.

[0335] Based on the cleaning robot 1000 detecting a liquid object on a surface to be cleaned on a travel path while moving along the travel path, at least one processor 1100 may change the travel path to avoid the liquid object.

[0336] The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" refers to a tangible device and does not include signals (e.g., electromagnetic waves), and the term "non-transitory storage medium" does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is stored temporarily. For example, the "non-transitory storage medium" may include a buffer for temporarily storing data.

[0337] According to an embodiment, a method according to various embodiments disclosed herein may be included in a computer program product and then provided. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc ROM (CD-ROM) or a universal serial bus (USB) flash drive), or may be distributed online through an app store (e.g., downloaded or uploaded) or directly between two user devices (e.g., a smart phone). In the case of online distribution, at least a part of the computer program product (e.g., a downloadable application) may be temporarily stored in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

Claims

1. A cleaning robot (1000), comprising: Cleaning module (1900); Traveling module (1200), configured to move the cleaning robot (1000) on a surface to be cleaned; Light emitting unit (1800); Infrared light sensor (1710); Visible light sensor (1720); At least one memory (1400) storing one or more instructions; And At least one processor (1100), configured to execute the one or more instructions stored in the memory (1400) to control the light emitting unit (1800) to emit infrared light to a detection area on the surface to be cleaned in front of the cleaning robot (1000), receive the infrared light reflected from the detection area through the infrared light sensor (1710), receive the visible light reflected from the detection area through the visible light sensor (1720), determine whether there is a liquid object in the detection area based on the intensity of the received infrared light and the visible light, control the traveling module (1200) to move the cleaning robot (1000) to avoid the liquid object based on determining that there is a liquid object in the detection area in front of the cleaning robot (1000), and control the cleaning module (1900) to clean the surface to be cleaned.

2. The cleaning robot according to claim 1, wherein, The at least one processor is further configured to calculate the reflectivity of the infrared light emitted to the detection area based on the intensity of the received infrared light, and determine whether there is a liquid object in the detection area in front based on the visible light and whether the calculated reflectivity is greater than or equal to a threshold reflectivity.

3. The cleaning robot according to claim 2, wherein, The light source of the light emitting unit is arranged at a height where the surface reflectivity of the liquid object in the detection area is greater than or equal to the threshold reflectivity.

4. The cleaning robot according to claim 1, wherein, The at least one processor is further configured to control the light emitting unit to emit light having an absorption wavelength determined based on the absorption characteristics of water as a peak wavelength to the detection area, and receive the infrared light having the absorption wavelength in the light reflected from the detection area through the infrared light sensor.

5. The cleaning robot according to claim 4, wherein, The range of the absorption wavelength is between 930 nm and 1030 nm.

6. The cleaning robot according to claim 4 or 5, wherein, The infrared light sensor includes a band-pass filter for blocking visible light and receiving infrared light having the absorption wavelength.

7. The cleaning robot according to any one of claims 1 to 6, wherein, The at least one processor is further configured to generate an infrared light image corresponding to the intensity of the received infrared light, generate a visible light image corresponding to the intensity of the received visible light, and determine whether there is a liquid object in the detection area in front of the cleaning robot (1000) based on the infrared light image and the visible light image.

8. The cleaning robot according to claim 7, wherein, The at least one processor is further configured to generate a composite image by combining the infrared light image and the visible light image, obtain the area of the liquid object in the visible light image by applying the composite image to a machine learning model, and determine whether there is a liquid object in the detection area in front of the cleaning robot (1000) based on the area of the liquid object in the visible light image.

9. The cleaning robot according to any one of claims 1 to 7, wherein, The at least one processor is further configured to output a notification that a liquid object exists in a detection area in front of the cleaning robot (1000) based on determining that a liquid object exists in the detection area in front of the cleaning robot (1000).

10. The cleaning robot according to any one of claims 1 to 8, wherein, The at least one processor is further configured to send, to a user device through a server, a notification that a liquid object exists in a detection area in front of the cleaning robot (1000) based on determining that a liquid object exists in the detection area in front of the cleaning robot (1000).

11. A method for controlling a cleaning robot to avoid a liquid object, the method comprising: Emit infrared light to a detection area on a surface to be cleaned in front of the cleaning robot; Receive, by an infrared light sensor of the cleaning robot, the infrared light reflected from the detection area; Receive, by a visible light sensor of the cleaning robot, the visible light reflected from the detection area; Determine whether a liquid object exists in the detection area based on the intensity of the received infrared light and the visible light; Based on determining that a liquid object exists in the detection area, control a travel module of the cleaning robot to move the cleaning robot to avoid the liquid object; And Control a cleaning module of the cleaning robot to clean the surface to be cleaned.

12. The method according to claim 11, wherein, Determining whether a liquid object exists in the detection area based on the intensity of the received infrared light and the visible light includes: Calculating a reflectivity of the infrared light emitted toward the detection area based on the intensity of the received infrared light; and Determining whether a liquid object exists in a detection area in front of the cleaning robot (1000) based on whether the visible light and the calculated reflectivity are greater than or equal to a threshold reflectivity.

13. The method according to claim 11, wherein, Emitting infrared light to a detection area on a surface to be cleaned in front of the cleaning robot includes: emitting light having an absorption wavelength determined based on the absorption characteristics of water as a peak wavelength to the detection area, and Receiving, by an infrared light sensor of the cleaning robot, the infrared light reflected from the detection area includes: receiving, by the infrared light sensor of the cleaning robot, the infrared light having the absorption wavelength in the light reflected from the detection area.

14. The method according to claim 13, wherein, The range of the absorption wavelength is between 930 nm and 1030 nm.

15. The method according to claim 13 or 14, wherein, The infrared light sensor includes a band-pass filter for blocking visible light and receiving infrared light having the absorption wavelength.

Images