Robotic vacuum cleaner and method for controlling same

Abstract

A robotic vacuum cleaner and a method for controlling same are disclosed. In particular, the robotic vacuum cleaner according to the present disclosure may include: a first sensor using ultrasonic waves; at least one second sensor related to a movement of the robotic vacuum cleaner; a memory storing information on a traveling path of the robotic vacuum cleaner; and a processor that: obtains first data related to a feature of a floor surface on the traveling path through the first sensor while the robotic vacuum cleaner moves along the traveling path; obtains second data related to the movement of the robotic vacuum cleaner through the at least one second sensor; obtains type information indicating the type of the floor surface by inputting a traveling data set including at least part of the first data and the second data to a trained neural network model; and controls the operation of the robotic vacuum cleaner on the basis of the type information.

Description

Robot vacuum cleaner and control method thereof

[0001] The present disclosure relates to a robot vacuum cleaner and a control method thereof, and more particularly, to provide a robot vacuum cleaner and a control method thereof capable of identifying a type of floor surface and performing an operation corresponding to the identified type of floor surface.

[0002] Recently, technological advancements in robotic vacuum cleaners capable of cleaning floors while autonomously following a path are accelerating. In particular, robotic vacuum cleaners capable of cleaning in an appropriate manner depending on the type of floor have recently been introduced.

[0003] However, in order for a robot vacuum cleaner to perform cleaning with high user satisfaction, it is important to accurately identify the type of floor surface and perform cleaning in a method appropriate to the identified type of floor surface.

[0004] Conventional techniques for detecting the type of floor surface include techniques using ultrasonic sensors, taking images of the floor surface, using cliff sensors that detect the height of a robot cleaner from the floor surface, and using wheel sensors that detect changes in the speed of a robot cleaner wheel.

[0005] However, when applying a technology using an ultrasonic sensor among the conventional technologies, one ultrasonic sensor is generally placed on the front left or front right due to space constraints caused by bulky parts such as casters (small wheels at the front), batteries, and drum brushes, and for cost reduction reasons. Therefore, with regard to the conventional technology using an ultrasonic sensor, there may be areas in the driving path where the ultrasonic sensor does not detect the type of floor surface (specifically, this will be described with reference to Fig. 1), and in particular, when driving in wet mode using a mop for water cleaning, there is a limitation that the carpet area not detected by the ultrasonic sensor may become wet with water.

[0006] Meanwhile, to overcome the limitations of the above conventional technology, there is a technology that gives priority to rules such as not lifting the mop for a certain period of time even if a floor without carpet is detected during wet mode driving. However, this technology has limitations, such as that it can only prevent the carpet from getting wet when the robot cleaner leaves the carpet area, and it is difficult to prevent the carpet from getting wet when the robot cleaner enters the carpet area.

[0007] Additionally, when identifying the type of floor surface using only ultrasonic sensors, it is pointed out that there is a limitation that tatami-mat carpets are misclassified as floors, causing the carpets to become wet when driving in wet mode.

[0008] Meanwhile, although auxiliary sensors such as image sensors and cliff sensors can be added to reinforce the ultrasonic sensor, this incurs a separate high cost, and even when auxiliary sensors such as image sensors and cliff sensors are used, the accuracy of floor type classification is not high, and there is a limitation that the carpet area can be detected only after the robot cleaner has climbed onto the carpet and traveled a certain distance (about 15 cm).

[0009] The present disclosure is intended to address the limitations of the prior art as described above, and the purpose of the present disclosure is to provide a robot vacuum cleaner and a control method thereof capable of efficiently and accurately identifying the type of floor surface using a learned neural network model and performing an operation suitable for the type of the identified floor surface.

[0010] According to one embodiment of the present disclosure for achieving the above-described object, the robot cleaner includes a first sensor using ultrasonic waves, at least one second sensor related to the movement of the robot cleaner, a memory for storing information on a driving path of the robot cleaner, a processor for obtaining first data related to a feature of a floor surface on the driving path through the first sensor while the robot cleaner moves along the driving path, obtaining second data related to the movement of the robot cleaner through the at least one second sensor, inputting a driving data set including at least a portion of the first data and the second data into a learned neural network model to obtain type information indicating a type of the floor surface, and controlling an operation of the robot cleaner based on the type information.

[0011] Meanwhile, the at least one second sensor may include at least one of an acceleration sensor, an angular velocity sensor, and a wheel sensor, and the type information may include information on whether a carpet exists on the floor surface along the driving path, a material of the floor surface, and at least one of a material of the carpet.

[0012] Meanwhile, the processor inputs only at least a portion of the second data among the first data and the second data into the neural network model to obtain the type information, identifies the type of the floor surface based on the first data, and if it is determined that a carpet exists on the floor surface based on the first data, controls the operation of the robot cleaner based on the identification result that a carpet exists on the floor surface, and if it is determined that no carpet exists on the floor surface based on the first data, controls the operation of the robot cleaner based on the type information.

[0013] Meanwhile, the system further includes at least one third sensor using infrared rays, and the processor obtains third data related to the color of the floor surface and including information on the reflectivity of the floor surface with respect to the infrared rays through the at least one third sensor, and inputs a driving data set including at least some of the first data, the second data, and the third data into a learned neural network model to obtain the type information, and the driving data set may include data related to shaking of the robot cleaner among the second data and the third data.

[0014] Meanwhile, the processor may include the third data in the driving data set if a value representing a change in data related to shaking of the robot cleaner is greater than or equal to a first threshold value and a value representing a change in the third data is greater than or equal to a second threshold value.

[0015] Meanwhile, the processor may convert a portion of the driving data set so that a value representing a change in data related to the shaking of the robot cleaner and a value representing a change in the third data have a larger value, and input the converted driving data set into the neural network model to obtain the type information.

[0016] Meanwhile, the device further includes a driving unit including a dry brush, a wet brush, and at least one motor, and the processor can control the driving unit to lift the wet brush while performing cleaning on the floor surface using the wet brush, if it is determined that a carpet exists on the floor surface based on the type information.

[0017] Meanwhile, if the processor determines that a carpet exists on the floor surface based on the type information while the robot cleaner is performing cleaning on the floor surface using the dry brush, the processor can control the driving unit to increase the suction power of the robot cleaner while performing cleaning on the carpet area.

[0018] Meanwhile, the processor can modify information about the driving path stored in the memory based on the type information, and control the driving unit to cause the robot cleaner to drive based on the information about the modified driving path.

[0019] Meanwhile, if the processor determines that a carpet of a first material exists on the floor surface based on the type information while cleaning the floor surface using the wet brush, the processor controls the driving unit to lift the wet brush while cleaning the carpet area where the carpet of the first material exists, and if the processor determines that a carpet of a second material different from the first material exists on the floor surface based on the type information while cleaning the floor surface using the wet brush, the processor modifies information about the driving path stored in the memory and controls the driving unit to drive the robot cleaner based on the information about the modified driving path.

[0020] Meanwhile, the wet brush includes a first wet brush arranged on a first side of the robot cleaner and a second wet brush arranged on a second side different from the first side, and the processor controls the driving unit to lift the first wet brush and not lift the second wet brush while performing cleaning on the composite area using the wet brush when a composite area in which a carpet exists on a floor surface corresponding to the first side and no carpet exists on a floor surface corresponding to the second side is identified based on the type information.

[0021] According to one embodiment of the present disclosure for achieving the above-described object, a method for controlling a robot cleaner includes the steps of: obtaining first data related to a feature of a floor surface on a driving path through a first sensor using ultrasonic waves while the robot cleaner moves along the driving path; obtaining second data related to a movement of the robot cleaner through at least one second sensor related to the movement of the robot cleaner while the robot cleaner moves along the driving path; inputting a driving data set including at least a portion of the first data and the second data into a learned neural network model to obtain type information indicating a type of the floor surface; and controlling an operation of the robot cleaner based on the type information.

[0022] Meanwhile, the at least one second sensor may include at least one of an acceleration sensor, an angular velocity sensor, and a wheel sensor, and the type information may include information on whether a carpet exists on the floor surface along the driving path, a material of the floor surface, and at least one of a material of the carpet.

[0023] Meanwhile, the step of obtaining the type information may include a step of inputting only at least a part of the second data among the first data and the second data into the neural network model to obtain the type information, and the step of controlling the operation of the robot cleaner may further include a step of identifying the type of the floor surface based on the first data, and a step of controlling the operation of the robot cleaner based on the identification result that the carpet exists on the floor surface when it is determined that a carpet exists on the floor surface based on the first data, and a step of controlling the operation of the robot cleaner based on the type information when it is determined that no carpet exists on the floor surface based on the first data.

[0024] Meanwhile, the control method of the robot cleaner further includes a step of obtaining third data including information on the reflectivity of the floor surface with respect to the infrared ray and related to the color of the floor surface through at least one third sensor using infrared ray, and the step of obtaining the type information includes a step of obtaining the type information by inputting a driving data set including at least some of the first data, the second data, and the third data into a learned neural network model, and the driving data set may include data related to shaking of the robot cleaner among the second data and the third data.

[0025] Meanwhile, the step of acquiring the type information may further include a step of including the third data in the driving data set if a value representing a change amount of data related to shaking of the robot cleaner is greater than or equal to a first threshold value and a value representing a change amount of the third data is greater than or equal to a second threshold value.

[0026] Meanwhile, the step of obtaining the type information may include a step of converting a portion of the driving data set so that a value representing a change amount of data related to the shaking of the robot cleaner and a value representing a change amount of the third data have a larger value, and a step of inputting the converted driving data set into the neural network model to obtain the type information.

[0027] Meanwhile, the step of controlling the operation of the robot cleaner may include a step of lifting the wet brush while cleaning the carpet area where the carpet exists, if it is identified that a carpet exists on the floor surface based on the type information while cleaning the floor surface using the wet brush.

[0028] Meanwhile, the step of controlling the operation of the robot cleaner may further include a step of increasing the suction power of the robot cleaner while performing cleaning on the carpet area, if it is determined that a carpet exists on the floor surface based on the type information while the robot cleaner performs cleaning on the floor surface using a dry brush.

[0029] Meanwhile, the step of controlling the operation of the robot cleaner may further include a step of modifying information about the driving path stored in the memory based on the type information and a step of controlling the robot cleaner to drive based on the information about the modified driving path.

[0030] Figure 1 is a drawing for explaining the operation of a robot vacuum cleaner according to the prior art.

[0031] FIG. 2 is a block diagram briefly illustrating the configuration of a robot vacuum cleaner according to one or more embodiments of the present disclosure;

[0032] FIG. 3 is a diagram illustrating a neural network model and a plurality of modules according to one or more embodiments;

[0033] FIG. 4 is a drawing illustrating one or more embodiments related to a floor surface type identification module according to the present disclosure;

[0034] FIG. 5 is a block diagram showing the configuration of a robot vacuum cleaner according to one or more embodiments of the present disclosure;

[0035] FIG. 6 is a drawing for explaining one or more embodiments related to a third sensor according to the present disclosure;

[0036] FIG. 7 is a drawing illustrating one or more embodiments related to the location and number of third sensors;

[0037] FIG. 8 is a block diagram showing the configuration of a robot vacuum cleaner according to one or more embodiments of the present disclosure;

[0038] FIG. 9 is a drawing for explaining a control process of a wet brush according to one or more embodiments, and

[0039] FIG. 10 is a flowchart illustrating a method for controlling a robot vacuum cleaner according to one or more embodiments of the present disclosure.

[0040] The present embodiments may be modified and have various embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0041] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0042] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0043] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0044] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0045] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0046] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0047] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0048] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0049] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0050] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0051] In the embodiments, a 'module' or 'part' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of 'modules' or 'parts' may be integrated into at least one module and implemented as at least one processor, except for a 'module' or 'part' that needs to be implemented as a specific hardware.

[0052] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0053] Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement the present disclosure.

[0054] Figure 1 is a drawing for explaining the operation of a robot vacuum cleaner according to the prior art.

[0055] As illustrated in Figure 1, a conventional robot cleaner can drive forward along a driving path. Furthermore, the conventional robot cleaner can identify the type of floor surface along its driving path using an ultrasonic sensor positioned on the front left side of the robot cleaner.

[0056] However, since the ultrasonic sensor can only detect information about the characteristics of the floor surface directly below where the ultrasonic sensor is located, it may be difficult to accurately identify the type of floor surface on the front right side of the robot cleaner with only one ultrasonic sensor placed on the front left side of the robot cleaner.

[0057] For example, as illustrated in FIG. 1, if there is a carpet on the right side of the driving path and no carpet on the left side of the driving path, a robot cleaner according to the prior art can only detect information about the characteristics of the floor surface of the left side, and thus can identify that there is no carpet on the floor surface of the right side either. In this case, when the robot cleaner drives along the driving path, the robot cleaner cannot perform cleaning suitable for the carpet area on the front right side, and in particular, when driving in wet mode using a mop to perform water cleaning, the carpet area (10) on the front right side of the robot cleaner may become wet.

[0058] In addition to the example of Fig. 1, conventional robot cleaners have a limitation in that areas may not be recognized as carpet areas when the robot cleaner leaves the carpet area, and thus areas may become wet by the mop when driving in wet mode.

[0059] In addition, conventional robot cleaners have limitations in that they cannot perform appropriate cleaning according to the type of floor surface, such as when the robot cleaner enters or leaves the carpet area at an angle (i.e., when one side of the robot cleaner enters or leaves the carpet area before the other side), or when a carpet of a special material, such as a tatami-mat carpet, exists on the floor surface, and they can make the carpet area wet.

[0060] Below, various embodiments for overcoming the limitations of the prior art as described above are described in detail.

[0061] FIG. 2 is a block diagram briefly illustrating the configuration of a robot vacuum cleaner (100) according to one or more embodiments of the present disclosure, and FIG. 3 is a diagram for explaining a neural network model (1000) and a plurality of modules according to one or more embodiments. Hereinafter, descriptions will be made with reference to FIG. 2 and FIG. 3 together.

[0062] A 'robot cleaner (100)' refers to a robot-based device that automatically cleans a floor surface (i.e., a surface to be cleaned). In particular, the robot cleaner (100) can autonomously drive along a driving path based on data acquired through various sensors, and capture (collect) contaminants such as dust, hair, and fine particles on the floor surface. The robot cleaner (100) can be implemented in various types depending on the configuration included in the robot cleaner (100) and the functions implemented through the robot cleaner (100), but there is no particular limitation on the type of the robot cleaner (100) according to the present disclosure.

[0063] As illustrated in FIG. 2, the robot cleaner (100) may include a first sensor, a second sensor, a memory (130), and a processor (140).

[0064] The "first sensor" is a sensor that utilizes ultrasound and may be referred to by terms such as "ultrasonic sensor." Specifically, the first sensor can obtain data on the characteristics of objects, distances to objects, etc., by emitting ultrasound and receiving ultrasound reflected by various objects placed on the floor or in the cleaning space.

[0065] In particular, the first sensor according to the present disclosure can acquire first data. The first sensor can acquire the first data at preset intervals (e.g., 20 ms), and the first data acquired at each interval can indicate the characteristics of the floor surface, which is the sensing target area, at the time when the first data is acquired. Here, the 'sensing target area' is the area to be sensed by the first sensor, and refers to a part of the front floor surface of the robot cleaner (100). The range of the sensing target area can be determined according to the position, number, specifications, etc. of the first sensor.

[0066] The term "first data" refers to data on the characteristics of the floor surface on the driving path of the robot cleaner (100) acquired through the first sensor, and may include at least one sensing value acquired through the first sensor. Meanwhile, in the present disclosure, the term "data" may be used to collectively refer to at least one "sensing value" acquired through the sensor. For example, the first data may be composed of binary values ​​including a value of 1 indicating that a carpet exists in the sensing target area, and a value of 0 indicating that no carpet exists in the sensing target area, i.e., a normal floor.

[0067] The first sensor may be positioned on the front left side of the robot cleaner (100) as illustrated in FIG. 1, and the number of first sensors may be one. While there are no particular limitations on the location and number of first sensors, the description of the present disclosure assumes that only one first sensor is positioned on the front left side of the robot cleaner (100) as illustrated in FIG. 1.

[0068] The term "second sensor" refers to a sensor related to the movement of the robot vacuum cleaner (100). There may be at least one second sensor, or one or more. Specifically, at least one second sensor may include at least one of an acceleration sensor, an angular velocity sensor, and a wheel sensor.

[0069] The acceleration sensor can obtain data on acceleration according to the movement of the robot cleaner (100). Specifically, the acceleration sensor can obtain sensing values ​​representing acceleration in the x-axis, y-axis, and z-axis directions while the robot cleaner (100) moves. The data obtained through the acceleration sensor can be used to obtain information on movement, stoppage, acceleration, deceleration, movement direction, shaking, and whether the robot cleaner (100) is maintained in balance.

[0070] The angular velocity sensor can obtain data on the angular velocity according to the rotational movement of the robot cleaner (100). The angular velocity sensor can include a gyroscope sensor. Specifically, the angular velocity sensor can obtain sensing values ​​such as the amount of change in the rotational movement of the robot cleaner (100), the roll, pitch, and yaw values ​​of the robot cleaner (100), etc. The data obtained through the angular velocity sensor can be used to obtain information on whether the robot cleaner (100) is rotating, the direction of rotation, the degree of rotation, the posture of the robot, etc.

[0071] The wheel sensor can obtain data on the movement of the wheel of the robot cleaner (100). Specifically, the wheel sensor can obtain sensing values ​​such as a linear velocity value and an angular velocity value indicating the rotational speed and direction of the wheel while the robot cleaner (100) moves, a value indicating the accumulated number of rotations of the wheel, and a duty value of power transmitted to a wheel motor connected to the wheel. The data obtained through the wheel sensor can be used to obtain information on the movement direction, movement distance, movement speed, etc. of the robot cleaner (100).

[0072] In addition, various sensors, such as a collision sensor that detects impact caused by the robot cleaner (100) colliding with an obstacle, may be included in the second sensor according to the present disclosure.

[0073] In particular, the second sensor according to the present disclosure can obtain second data. The second sensor can obtain second data at preset intervals. Here, the 'second data' refers to data related to the movement of the robot cleaner (100) obtained through at least one second sensor, and may include at least one sensing value obtained through at least one second sensor. Here, the data related to the movement of the robot cleaner (100) may refer to data on changes in sensing values ​​caused by the movement of the robot cleaner (100) when moving in an area having a difference in height from the floor surface, such as a carpet area.

[0074] For example, the second data may include at least some of the sensing values ​​of the acceleration sensor, the angular velocity sensing values, and the sensing values ​​of the wheel sensor, as exemplified above. In addition, any data related to the movement of the robot cleaner (100) may be included in the second data according to the present disclosure.

[0075] At least one instruction regarding the robot cleaner (100) may be stored in the memory (130). In addition, an O / S (Operating System) for operating the robot cleaner (100) may be stored in the memory (130). In addition, various software programs or applications for operating the robot cleaner (100) according to various embodiments of the present disclosure may be stored in the memory (130). In addition, the memory (130) may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0076] Specifically, various software modules for operating the robot cleaner (100) according to various embodiments of the present disclosure may be stored in the memory (130), and the processor (140) may control the operation of the robot cleaner (100) by executing the various software modules stored in the memory (130). That is, the memory (130) is accessed by the processor (140), and data reading / recording / modifying / deleting / updating, etc. may be performed by the processor (140).

[0077] Meanwhile, in the present disclosure, the term memory (130) may be used to mean a memory (130), a ROM, a RAM in a processor (140), or a memory card (e.g., a micro SD card, a memory stick) mounted in a robot cleaner (100).

[0078] In particular, in one or more embodiments, the memory (130) may store information about a neural network model (1000), a learning data set of the neural network model (1000), information about a driving path of the robot cleaner (100), type information about the type of floor surface, etc. The information about the driving path stored in the memory (130) may be updated based on a user input, type information about the type of floor surface, etc. In addition, various information necessary within the scope of achieving the purpose of the present disclosure may be stored in the memory (130), and the information stored in the memory (130) may be updated as received from an external device or input by a user.

[0079] The processor (140) controls the overall operation of the robot cleaner (100). Specifically, the processor (140) is connected to a configuration of the robot cleaner (100) including a first sensor, at least one second sensor, and a memory (130), and can control the overall operation of the robot cleaner (100) by executing at least one instruction stored in the memory (130) as described above.

[0080] The processor (140) may be implemented in various ways. For example, the processor (140) may be implemented as at least one of an application-specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), and a digital signal processor (DSP). Meanwhile, the term "processor (140)" in the present disclosure may be used to mean a central processing unit (CPU), a graphic processing unit (GPU), and a microprocessor unit (MPU).

[0081] In particular, in one or more embodiments, the processor (140) can accurately identify the type of floor surface and perform an operation appropriate for the identified type of floor surface. Specifically, the processor (140) can perform processes related to various embodiments according to the present disclosure using a plurality of modules. As illustrated in FIG. 3, the plurality of modules may include a preprocessing module (141), a postprocessing module (142), a control module (143), and a learning module (144).

[0082] The plurality of modules may be implemented as hardware modules or software modules, and at least some of the modules may include a neural network model (1000). For convenience of explanation, the following description will assume that all of the plurality of modules are implemented through the memory (130) and processor (140) of the robot cleaner (100). However, depending on the embodiment, at least some of the plurality of modules may be implemented by an external device or server.

[0083] The processor (140) can acquire first data related to the characteristics of the floor surface along the driving path through the first sensor while the robot cleaner (100) moves along the driving path, and can acquire second data related to the movement of the robot cleaner (100) through at least one second sensor. Since the process of acquiring the first data and the second data through the first sensor and the second sensor, respectively, has been described above, a duplicate description of the same content will be omitted.

[0084] The processor (140) can input a driving data set including at least a portion of the first data and the second data into a learned neural network model (1000) to obtain type information indicating the type of the floor surface. Specifically, the processor (140) can obtain the type information using a preprocessing module (141), a neural network model (1000), and a postprocessing module (142).

[0085] The 'preprocessing module (141)' refers to a module that can acquire a driving data set based on data acquired through the first sensor, the second sensor, and the third sensor (the third sensor will be described later in the description of FIGS. 5 to 7).

[0086] Here, the 'driving data set' refers to a set of data that is input to the neural network model (1000) after being processed by the preprocessing module (141), and may include at least a portion of data acquired through a sensor.

[0087] The two are distinguished in that the learning data described below is used for learning the neural network model (1000), while the driving data set includes data acquired in real time through the first sensor, the second sensor, and the third sensor while the robot cleaner (100) is driving. However, the types of data or sensed values ​​included in the driving data set are configured identically to those of the learning data.

[0088] As illustrated in FIG. 3, when first data is received from a first sensor and second data is received from a second sensor, the preprocessing module (141) can process (or process) at least a portion of the entire data including the first data and the second data to obtain a driving data set.

[0089] For example, the preprocessing module (141) can combine sensing values ​​of a certain period (e.g., 100 ms when the data collection cycle is 20 ms) acquired through the first sensor and the second sensor, determine which sensing values ​​acquired through the first sensor and the second sensor to include in the driving data set, and can obtain the driving data set by assigning weights to the sensing values. In addition, data can be encoded or embedded according to the purpose of the neural network model (1000).

[0090] In the present disclosure, a "neural network model (1000)" refers to an artificial intelligence model including a neural network trained to acquire information indicating the type of floor surface based on a driving data set. As illustrated in FIG. 3, when a driving data set is received from the preprocessing module (141), the neural network model (1000) can identify the type of floor surface corresponding to the driving data set and output a probability value indicating which type of floor surface the driving data set corresponds to.

[0091] For example, the neural network model (1000) can be implemented as a CNN (Convolutional Neural Network), a DNN (Deep Neural Network), an RNN (Recurrent Neural Network), an RBM (Restricted Boltzmann Machine), a DBN (Deep Belief Network), a BRDNN (Bidirectional Recurrent Deep Neural Network), etc., but is not limited to these examples.

[0092] A "training data set" refers to a collection of data used for training a neural network model (1000). Like the driving data set, the training data set may include at least some of the data acquired through the first sensor, second sensor, and third sensor. In particular, the training data set may include a carpet entry data set and a threshold driving data set.

[0093] The carpet entry data set may include data for various cases, such as straight, corner right, corner left, diagonal right, and diagonal left, based on the form in which the robot cleaner (100) steps onto the carpet. The threshold driving data set may include data for various single-height thresholds, such as 3 mm, 5 mm, and 7 mm, as well as composite height thresholds, which are a combination of multiple heights, to enable the neural network model (1000) to classify the threshold as a floor surface without carpet. In addition, the learning data set may include data for various carpet materials, such as short pile, long pile, mats, and tatami mats.

[0094] The neural network model (1000) can output various types of probability values ​​depending on the type of learning data set used for learning and the predefined class (domain or category). For example, the neural network model (1000) can output a probability value for whether or not a carpet exists on the floor, a probability value for what material the floor is made of, or a probability value for what material the carpet on the floor is made of. Meanwhile, the neural network model (1000) can be implemented as a plurality of models trained to output output values ​​for each of the various types described above.

[0095] In particular, since the neural network model (1000) is trained based on learning data that includes not only first data on the characteristics of the floor surface acquired using ultrasound but also data related to the movement of the robot cleaner (100), it can be trained to obtain an output value for the type of floor surface by reflecting information on the movement of the robot cleaner (100) when it goes up or down the carpet.

[0096] Meanwhile, in the description of the present disclosure, it will be explained on the premise that the neural network model (1000) is implemented as an on-device in the robot cleaner (100), but the neural network model (1000) may be included in an external server, and the robot cleaner (100) may obtain the output value of the neural network model (1000) by receiving it through communication with the external server.

[0097] The 'post-processing module (142)' refers to a module capable of acquiring type information based on the output of the neural network model (1000). As illustrated in FIG. 3, when information on a probability value is received from the neural network model (1000), the post-processing module (142) can output type information indicating the type of floor surface corresponding to the driving data set based on the probability value of which type of floor surface the driving data set corresponds to.

[0098] 'Type information' is used as a general term to refer to information indicating the type of floor surface on the driving path of a robot cleaner (100). Specifically, the type information may include information on whether a carpet exists on the floor surface on the driving path, the material of the floor surface, and at least one of the material of the carpet.

[0099] For example, if the probability value for the probability that a carpet exists on the floor is greater than or equal to a threshold value and greater than the probability value for the probability that a carpet does not exist on the floor, the post-processing module (142) can obtain type information indicating that a carpet exists on the floor. Meanwhile, if the neural network model (1000) is implemented as a plurality of neural network models (1000) as described above, the post-processing module (142) may combine the probability values ​​output by each of the plurality of neural network models (1000) using a voting technique or the like, and obtain type information corresponding to the class most selected by the plurality of neural network models (1000).

[0100] Various examples of type information and various embodiments related to the control of the operation of the robot cleaner (100) according to the type information are described in more detail with reference to FIGS. 8 and 9.

[0101] Meanwhile, in the above description, for the sake of detailed explanation, it is assumed that the preprocessing module (141), the neural network model (1000), and the postprocessing module (142) are implemented as separate modules and models. However, at least some of the preprocessing module (141), the neural network model (1000), and the postprocessing module (142) may be implemented as a single neural network model (1000). For example, the neural network model (1000) may not output a probability value for the type of the floor surface, but may also output type information by performing postprocessing on the probability value.

[0102] The 'learning module (144)' refers to a module capable of training a neural network model (1000). Specifically, the learning module (144) can train a neural network model (1000) using a supervised learning method based on a training data set including labels. In addition, the neural network model (1000) may also be trained using unsupervised learning, semi-supervised learning, or reinforcement learning.

[0103] Meanwhile, the learning module (144) may update the learning data set with a combination of the driving data set and type information acquired while the robot cleaner (100) is driving along the driving path, and retrain the neural network model (1000) based on the updated learning data set.

[0104] Once the type information is acquired, the processor (140) can control the operation of the robot cleaner (100) based on the type information. Specifically, the processor (140) can control the operation of the robot cleaner (100) using the control module (143).

[0105] The 'control module (143)' refers to a module that can control the operation of the robot cleaner (100) based on type information. As illustrated in FIG. 3, when type information is received from the post-processing module (142), the control module (143) obtains a control signal for controlling the operation of the robot cleaner (100) based on the type information, and can control the operation of the robot cleaner (100) based on the obtained control signal.

[0106] For example, if it is identified that a carpet exists on the floor based on the type information, the processor (140) can control to lift the wet brush while performing cleaning on the carpet area where the carpet exists, and can also control the robot cleaner (100) to bypass the carpet area by changing the driving path.

[0107] Various embodiments related to controlling the operation of the robot cleaner (100) based on type information are described in more detail with reference to FIGS. 8 and 9.

[0108] According to one or more embodiments described above with reference to FIGS. 2 and 3, the robot cleaner (100) trains a neural network model (1000) using not only data on the characteristics of the floor surface acquired using ultrasonic waves but also data related to the movement of the robot cleaner (100), and efficiently and accurately identifies the type of the floor surface using the trained neural network model (1000) and performs an operation suitable for the type of the identified floor surface.

[0109] In particular, the robot cleaner (100) can classify carpet entry cases and carpet materials with high accuracy, which could not be accurately classified when using only ultrasonic sensors, and also, since the sensors basically used for autonomous driving of the robot cleaner (100) are also used to identify floor types, the type of floor can be accurately identified with high efficiency and low cost.

[0110] For example, even if there is a carpet in the right area of ​​the driving path and no carpet in the left area of ​​the driving path as shown in FIG. 1, the robot cleaner (100) according to the present disclosure can identify the carpet area with high accuracy because it identifies the type of floor surface not only by relying on the ultrasonic sensor but also by using data related to the movement of the robot cleaner (100).

[0111] FIG. 4 is a drawing for explaining one or more embodiments related to a floor surface type identification module (145) according to the present disclosure.

[0112] In the embodiments of FIGS. 2 and 3, the first data and the second data are input into the neural network model (1000). However, as shown in FIG. 4, the processor (140) may also obtain type information by inputting only at least a portion of the second data among the first data and the second data into the neural network model (1000).

[0113] That is, in the embodiment of FIG. 4, the first data is not input to the neural network model (1000) but is input to the floor surface type identification module (145), and only the second data can be input to the neural network model (1000).

[0114] The 'floor surface type identification module (145)' refers to a module capable of identifying the floor surface type based on the first data acquired through the first sensor. As illustrated in FIG. 4, when the first data is received from the first sensor, the floor surface type identification module (145) can obtain an identification result for the floor surface type using only the first data, without using the second data and the output of the neural network model (1000).

[0115] As described above, the first data may be composed of binary values ​​including a value of 1 indicating that a carpet exists in the sensing target area and a value of 0 indicating that a carpet does not exist in the sensing target area. In addition, the floor surface type identification module (145) may identify that a carpet exists in the sensing target area if the value included in the first data is 1, and may identify that a carpet does not exist in the sensing target area if the value included in the first data is 0.

[0116] If it is determined that a carpet exists on the floor based on the first data, the processor (140) can control the operation of the robot cleaner (100) based on the determination result that a carpet exists on the floor. In other words, if it is determined that a carpet exists on the floor based on the first data, it can be said that it is determined that a carpet exists on the floor without using the neural network model (1000), and therefore, the processor (140) can control the operation of the robot cleaner (100) based on the determination result that a carpet exists on the floor without using the output of the neural network model (1000).

[0117] If it is determined that there is no carpet on the floor based on the first data, the processor (140) can control the operation of the robot cleaner (100) based on the type information. In other words, if it is determined that there is no carpet on the floor based on the first data, it can be said that there is still a possibility that there is a carpet on the floor when the neural network model (1000) is used together, and therefore, the processor (140) can identify the type of the floor based on the type information obtained using the neural network model (1000) and control the operation of the robot cleaner (100).

[0118] Meanwhile, in FIG. 4, the floor type identification module (145) uses the first data and the neural network model (1000) uses the second data. However, depending on the embodiment, the process in which the neural network model (1000) uses the second data may be omitted. That is, the processor (140) identifies the type of the floor based on the first data before obtaining type information using the neural network model (1000), and if it is determined that a carpet exists on the floor based on the first data, the processor (140) may not perform the process of obtaining type information based on the second data.

[0119] If the type of the floor surface is identified based on the first data before obtaining the type information using the neural network model (1000), the total amount of computation can be reduced, allowing the type of the floor surface to be identified in a more efficient manner. On the other hand, if the type of the floor surface is identified based on the first data after obtaining the type information using the neural network model (1000), the operation process of the neural network model (1000) is always performed regardless of the identification result based on the first data, so there is an advantage in that the process of identifying the type of the floor surface based on the first data can be easily added without significantly changing the algorithm according to the configuration of FIG. 3.

[0120] FIG. 5 is a block diagram showing the configuration of a robot cleaner (100) according to one or more embodiments of the present disclosure, FIG. 6 is a diagram for explaining one or more embodiments related to a third sensor according to the present disclosure, and FIG. 7 is a diagram for explaining one or more embodiments related to the position and number of third sensors. Hereinafter, description will be made with reference to FIGS. 5 to 7 together.

[0121] As illustrated in FIG. 5, the robot cleaner (100) may further include a third sensor in addition to the first sensor, the second sensor, the memory (130), and the processor (140).

[0122] The "third sensor" is a sensor that utilizes infrared light and may also be referred to as an "infrared sensor." Specifically, the third sensor emits infrared light and receives the infrared light reflected back from the floor. Since the degree of infrared reflection may vary depending on the color of the floor, the third sensor can obtain information about the floor's reflectivity based on the intensity of the received infrared light.

[0123] In the description of the present disclosure, data acquired through a third sensor is referred to as "third data." The third data includes information on the reflectivity of a floor surface and may be related to the color of the floor surface. Specifically, since the reflectivity for infrared rays may vary depending on the color of the floor surface, information on the reflectivity of the floor surface may indicate the color of the floor surface. For example, the third data may indicate information on the difference in reflectivity between a carpeted area on the floor surface and a non-carpeted area, and may also indicate information on the difference in reflectivity depending on the material of the carpet.

[0124] That is, the infrared sensor mounted on the robot cleaner (100) can generally be used to obtain information on the distance between the robot cleaner (100) and the floor and to determine whether the robot cleaner (100) has fallen, but in the present disclosure, it can be used in the process of obtaining information on the reflectivity of the floor and obtaining third data related to the color of the floor.

[0125] The third sensor may be at least one, that is, one or more. As illustrated in FIG. 7, the at least one third sensor may include a third sensor (151) positioned on the front left side of the robot cleaner (100), a third sensor (152) positioned on the front right side, a third sensor (153) positioned on the left side, and a third sensor (154) positioned on the right side. FIG. 7 illustrates the positions of the third sensors based on a viewpoint looking at the robot cleaner (100) from above, and in reality, at least one third sensor may be positioned on the bottom surface of the robot cleaner (100).

[0126] When third data related to the color of the floor surface is acquired through at least one third sensor, the processor (140) can input a driving data set including at least some of the first data, the second data, and the third data into the trained neural network model (1000) to acquire type information. That is, the processor (140) can train the neural network model (1000) based on the training data set including not only the first data and the second data but also the third data, and accordingly, the neural network model (1000) can acquire type information by reflecting not only the first data and the second data but also the third data related to the color of the floor surface.

[0127] More specifically, as illustrated in FIG. 6, the processor (140) inputs the first data, the second data, and the third data into the preprocessing module (141), and can obtain type information through the preprocessing module (141), the neural network model (1000), and the postprocessing module (142). However, since the preprocessing module (141), the neural network model (1000), and the postprocessing module (142) have been described above, a duplicate description of the same content will be omitted.

[0128] Meanwhile, the third data acquired through the third sensor using infrared rays may result in false detection if the color change of the floor or carpet is severe. Therefore, the processor (140) may not directly include the third data in the driving data set and input it into the neural network model (1000), but may perform preprocessing on the third data and then input it into the driving data set and input it into the neural network model (1000). Here, the driving data set may include data related to the shaking of the robot cleaner (100) among the second data and the third data. Here, the 'data related to the shaking' may include the pitch value and roll value of the gyro sensor among the angular velocity sensors.

[0129] For example, if there is little change in the data related to the shaking of the robot cleaner (100) among the second data, but only a large change in the third data related to the color change of the floor surface, this does not indicate that there is a carpet area with a height difference from the floor surface, but rather that there is an area where the color change of the floor surface has drastically changed.

[0130] On the other hand, if the change in the data related to the shaking of the robot cleaner (100) among the second data is large and the change in the third data related to the color change of the floor surface is also large, this does not indicate that there is an area where the color change of the floor surface has drastically changed, but rather that there is an area of ​​the carpet that has a height difference from the floor surface.

[0131] Accordingly, the processor (140) can exclude the possibility of false detection of the floor type due to color change by not including third data indicating that there is an area where the color change of the floor surface has drastically changed in the driving data set.

[0132] In one or more embodiments, if a value representing a change amount of data related to shaking of the robot cleaner (100) is greater than or equal to a first threshold value and a value representing a change amount of third data is greater than or equal to a second threshold value, the processor (140) may include the third data in the driving data set. On the other hand, even if the value representing a change amount of the third data is greater than or equal to the second threshold value, if the value representing a change amount of the data related to shaking of the robot cleaner (100) is less than the first threshold value, the processor (140) may not include the third data in the driving data set.

[0133] Meanwhile, in one or more embodiments, the processor (140) may convert a portion of a driving data set such that a value representing a change amount of data related to shaking of the robot cleaner (100) and a value representing a change amount of third data have larger values, using a mathematical equation such as the following mathematical equation 1, and input the converted driving data set into a neural network model (1000) to obtain type information.

[0134] [Mathematical Formula 1]

[0135] {std(gyro_pitch)+std(gyro_roll)}*{std(IR1)+std(IR2)+std(IR3)+std(IR4)}

[0136] In mathematical expression 1, std refers to the standard deviation, gyro_pitch and gyro_roll refer to the pitch value and roll value of the gyro sensor among the angular velocity sensors, and IR1, IR2, IR3, and IR4 refer to the sensing values ​​obtained through each of the four third sensors.

[0137] The operation value of mathematical expression 1 may have a relatively higher value (e.g., 300 to 3000) when the robot cleaner (100) goes up or down the carpet, and may have a relatively lower value (e.g., less than 300) when the robot cleaner (100) does not pass over the carpet and there is simply a change in the color of the floor surface. In addition to mathematical expression 1, various mathematical expressions that can convert a part of the driving data set so that the larger the value representing the amount of change in data related to the shaking of the robot cleaner (100) and the value representing the amount of change in the third data, the larger the value, may be applied to the present disclosure.

[0138] Meanwhile, the above described embodiment in which the processor (140) inputs at least a part of the third data itself acquired through the third sensor into the learned neural network model (1000), the processor (140) may also acquire information on the color of the floor surface based on information on the reflectivity of the floor surface included in the third data, and input information on the color of the floor surface into the learned neural network model (1000).

[0139] According to one or more embodiments described above with reference to FIGS. 5 to 7, the robot cleaner (100) trains a neural network model (1000) using not only data on the characteristics of the floor surface acquired using ultrasonic waves and data related to the movement of the robot cleaner (100), but also data related to the color of the floor surface acquired using infrared rays, and can accurately identify the type of the floor surface using the trained neural network model (1000).

[0140] In particular, according to the prior art, when there is a carpet in the right area of ​​the driving path as illustrated in FIG. 1 and there is no carpet in the left area of ​​the driving path, the right side of the robot cleaner (100) without an ultrasonic sensor goes up and down the carpet, so there is a possibility that the robot cleaner (100) may identify that the robot cleaner (100) is going up and down a threshold. On the other hand, by using data related to the color of the floor surface acquired using infrared rays according to the present disclosure, even in a case as illustrated in FIG. 1, it is possible to identify that there is a carpet area on the floor surface without identifying that the robot cleaner (100) is going up and down a threshold.

[0141] Furthermore, the robot cleaner (100) can prevent an area where the color of the floor surface changes rapidly from being mistakenly recognized as a carpet area by preprocessing the third data using data related to the shaking of the robot cleaner (100) among the second data.

[0142] FIG. 8 is a block diagram showing the configuration of a robot vacuum cleaner (100) according to one or more embodiments of the present disclosure, and FIG. 9 is a diagram for explaining a control process of a wet brush according to one or more embodiments. Hereinafter, description will be made with reference to FIG. 8 and FIG. 9 together.

[0143] As illustrated in FIG. 8, the robot cleaner (100) may further include a brush (160), a driving unit (170), a communication unit (180), an input unit (190), and an output unit (195), in addition to a first sensor, a second sensor, a memory (130), a processor (140), and a third sensor. The configurations illustrated in FIGS. 2, 5, and 8 are merely exemplary, and it is to be understood that new configurations may be added or some configurations may be omitted in addition to the configurations illustrated in FIGS. 2, 5, and 8 when implementing the present disclosure.

[0144] The brush (160) can sweep away contaminants placed on the floor so that the suction unit of the robot cleaner (100) can easily capture the contaminants. Specifically, it can include a dry brush and a wet brush.

[0145] A "dry brush" refers to a brush capable of cleaning a floor without getting wet. Specifically, a dry brush can sweep away contaminants such as dust placed on the floor, allowing the suction unit of a robot cleaner (100) to easily capture the contaminants. The operating mode for cleaning using a dry brush may be referred to as a "dry mode."

[0146] A 'wet brush' refers to a brush that can clean a floor surface with water while it is wet, and may be referred to by terms such as a 'wet mop'. For example, a wet brush may receive water from a user or a water tank, detach contaminants adsorbed on a floor surface while it is wet, and sweep away the detached contaminants so that the suction unit of a robot cleaner (100) can easily capture the detached contaminants. An operation mode in which cleaning is performed using a wet brush may be referred to as a 'wet mode'.

[0147] When the robot cleaner (100) includes both a dry brush and a wet brush and is implemented so as to be able to control the position of the wet brush, the robot cleaner (100) can perform cleaning without using the wet brush when operating in dry mode, and can perform water cleaning using the wet brush by automatically controlling the position of the wet brush when operating in wet mode.

[0148] The driving unit (170) includes at least one motor and can control the operation of the robot cleaner (100). Specifically, the driving unit (170) can include a suction motor, a wet brush control motor, a wheel control motor, etc. At least one motor included in the driving unit (170) can be implemented as various types of motors such as a DC motor (Direct Current electric motor), an AC motor (Alternating Current electric motor), and a BLDC (brushless DC electric motor).

[0149] A "suction motor" refers to a motor capable of generating suction pressure. Specifically, when a control signal is received from a processor (140) and power is supplied from a power supply, the impeller can rotate by driving the suction motor. Suction pressure is generated by the rotation of the impeller, and air containing contaminants can be sucked into the suction port of the robot cleaner (100) by this suction pressure. Meanwhile, as the speed of the suction motor increases, the suction pressure can increase.

[0150] A 'wet brush control motor' refers to a motor capable of controlling the position of a wet brush. Specifically, the processor (140) can control the position of the wet brush by controlling the wet brush control motor connected to the wet brush.

[0151] For example, the processor (140) may control the wet brush motor to lower the position of the wet brush so that the wet brush can contact the floor surface in order to clean the floor surface using the wet brush, and further, the processor (140) may control the wet brush motor to raise the position of the wet brush so that the wet brush does not contact the floor surface in order to not clean the floor surface using the wet brush.

[0152] The 'wheel control motor' can control the operation of the wheel included in the robot cleaner (100). Specifically, the wheel control motor can control the direction of movement and speed of the wheel included in the robot cleaner (100) by controlling the direction of rotation and speed of the wheel. When the robot cleaner (100) includes two wheels, a left wheel and a right wheel, the wheel control motor can include a left wheel control motor and a right wheel control motor, and the left wheel control motor and the right wheel control motor can control the direction of rotation and speed of the left wheel and the right wheel, respectively.

[0153] As described above, the processor (140) can control the operation of the robot cleaner (100) based on the type information. Hereinafter, various embodiments in which the processor (140) controls the dry brush, wet brush, and driving unit (170) based on the type information will be described.

[0154] In one or more embodiments, when a carpet is identified on the floor surface based on type information while cleaning the floor surface using a wet brush, the processor (140) may control the driving unit (170) (specifically, the wet brush control motor) to lift the wet brush while cleaning the carpet area where the carpet is present. Meanwhile, the processor (140) may also control the driving unit (170) to lower the wet brush when the robot cleaner (100) leaves the carpet area.

[0155] In other words, the processor (140) can control the robot cleaner (100) to raise the wet brush before entering the carpet area while driving in wet mode to prevent the carpet from getting wet, and to lower the mop again to clean the floor when leaving the carpet area.

[0156] In one or more embodiments, when the robot cleaner (100) is performing cleaning on the floor using a dry brush and it is determined that a carpet exists on the floor based on the type information, the processor (140) may control the driving unit (170) (specifically, the suction motor) to increase the suction power of the robot cleaner (100) while performing cleaning on the carpet area.

[0157] In other words, the processor (140) can control the robot cleaner (100) to increase the suction power when it enters a carpet area while driving in dry mode to increase the carpet cleaning effect, and to lower the suction power again when it leaves the carpet area.

[0158] In one or more embodiments, the processor (140) may modify information about a driving path stored in the memory (130) based on the type information, and control the driving unit (170) (specifically, the wheel control motor) to cause the robot cleaner (100) to drive based on the information about the modified driving path.

[0159] Specifically, the type information acquired using the neural network model (1000) may include information on whether the robot cleaner (100) is entering or leaving a carpet area, and the direction of entry or exit. Furthermore, the processor (140) may modify the driving path in such a way that the cleaning operation at the boundary of the carpet area is effectively and accurately changed based on the identified entry and exit directions of the carpet according to the type information.

[0160] For example, if it is predicted that one wheel of the robot cleaner (100) will touch the edge of a carpet area along the driving path, the processor (140) may modify the driving path so that the robot cleaner (100) moves only on the floor surface or drives only on the carpet area. As another example, if it is predicted that the robot cleaner (100) will enter or leave the carpet area at an angle along the driving path, the processor (140) may modify the driving path so that the robot cleaner (100) enters or leaves the carpet area in a non-angled direction. As another example, if the automatic raising and lowering of the wet brush is not implemented, the processor (140) may modify the driving path so that the robot cleaner (100) bypasses the carpet area.

[0161] Meanwhile, the type information obtained using the neural network model (1000) may include information about the material of the carpet, and the processor (140) may control the driving unit (170) based on the information about the material of the carpet. More specifically, the type information obtained using the neural network model (1000) may include information about whether the carpet is made of long hair that is longer than a critical length or short hair that is shorter than the critical length, and the processor (140) may control the driving unit (170) based on information about the length of the carpet hair according to the type information.

[0162] In one or more embodiments, when a carpet of a first material is identified as being present on the floor surface based on type information while cleaning the floor surface using a wet brush, the processor (140) may control the driving unit to lift the wet brush while cleaning the carpet area where the carpet is present. Here, the term “first material” is used as a general term for a material of a carpet that enables efficient cleaning when the wet brush is lifted, and may be particularly related to the length of the carpet hair, as described below.

[0163] Specifically, when the length of the carpet existing on the floor surface is identified as being less than a threshold length based on the type information while cleaning the floor surface using a wet brush, the processor (140) can control the driving unit (170) to lift the wet brush while cleaning the carpet area where the carpet exists.

[0164] In other words, if the carpet hair length is identified as short, the wet brush may not touch the carpet when lifted, so the processor (140) can control the driving unit (170) to lift the wet brush while cleaning the carpet area where the carpet exists.

[0165] In one or more embodiments, when a carpet of a second material different from a first material is identified on the floor surface based on type information while cleaning the floor surface using a wet brush, information about a travel path stored in a memory may be modified, and a driving unit may be controlled to allow the robot cleaner to travel based on the information about the modified travel path. Here, the term "second material" is used as a general term for the material of a carpet that is difficult to clean efficiently even when a wet brush is lifted, and as described below, may be particularly related to the length of the carpet hair.

[0166] Specifically, when the length of the carpet existing on the floor is identified as being greater than a threshold length based on type information while cleaning the floor using a wet brush, the processor (140) can modify information on the driving path stored in the memory (130) and control the driving unit (170) to cause the robot cleaner (100) to drive based on the information on the modified driving path.

[0167] In other words, if the carpet fur is identified as long, the processor (140) may modify the driving path to include a location for detaching the wet brush or to bypass the carpet area, as the wet brush may still touch the carpet even when lifted.

[0168] Meanwhile, the type information acquired using the neural network model (1000) may include information indicating the type of the floor surface corresponding to each of the first side and the second side of the driving path. In addition, the wet brush may include a first wet brush arranged on the first side of the robot cleaner (100) and a second wet brush arranged on a side different from the first side. Here, the first side refers to one side of the robot cleaner (100), and the second side refers to another side of the robot cleaner (100) different from the first side.

[0169] Referring to the example of FIG. 9, the first side may be one of the left side and the right side of the robot cleaner (100), and the second side may be another side of the left side and the right side of the robot cleaner (100) that is different from the first side. However, in the present disclosure, there is no particular limitation on the position of the first side and the position of the second side, and there is no particular limitation on the relationship between the position of the first side and the position of the second side.

[0170] In one or more embodiments, if a composite area is identified based on the type information, in which a carpet exists on a floor surface corresponding to a first side and a carpet does not exist on a floor surface corresponding to a second side, the processor (140) may control the driving unit (170) to lift the first wet brush (161) and not to lift the second wet brush (162) while performing cleaning on the composite area using the wet brush.

[0171] Referring to the example of FIG. 9, when the floor surface types of the left area and the right area of ​​the driving path of the robot cleaner (100) are different, the processor (140) controls the driving unit to lift one of the wet brushes arranged on the left area and the wet brushes arranged on the right area and not lift the other while performing cleaning on the floor surface, thereby effectively performing a cleaning operation suitable for the floor surface types of the left area and the right area, respectively.

[0172] Meanwhile, the type information obtained using the neural network model (1000) may include information indicating the material of the floor surface. Furthermore, the processor (140) may identify the material of the floor surface based on the type information and control the driving unit (170) based on the identified material of the floor surface. For example, if the material of the floor surface is identified as wood, the processor (140) may reduce the amount of water supplied to the wet brush compared to cases where the material of the floor surface is vinyl, marble, or tile, thereby minimizing damage to the wood floor surface caused by water.

[0173] Meanwhile, the processor (140) can distinguish multiple areas of the floor based on the material of the identified floor. For example, the processor (140) can distinguish a first area where a carpet exists and a second area where no carpet exists by identifying a boundary where the material of the identified floor changes. Accordingly, even when the accuracy of distinguishing the floor may be low, such as when obstacles are placed on the floor at intervals similar to the width of a door, the multiple areas of the floor can be accurately distinguished, and appropriate cleaning can be performed for each of the multiple areas of the floor.

[0174] Meanwhile, the processor (140) may store information about a location identified as a carpet area in the memory (130). Specifically, the processor (140) may update information about a driving route stored in the memory (130) by modifying the driving route based on the location identified as a carpet area.

[0175] The communication unit (180) includes a circuit and can perform communication with an external device. Specifically, the processor (140) can receive various data or information from an external device connected via the communication unit (180) and can also transmit various data or information to the external device.

[0176] The communication unit (180) may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra-Wide Band). Specifically, the WiFi module and the Bluetooth module may each perform communication in the WiFi or Bluetooth manner. When using a WiFi module or a Bluetooth module, various connection information, such as an SSID, may be first transmitted and received, and then communication may be established using this, after which various pieces of information may be transmitted and received.

[0177] In addition, the wireless communication module can perform communication according to various communication standards such as IEEE, Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), 5G (5th Generation), etc. And, the NFC module can perform communication in the NFC (Near Field Communication) method using the 13.56MHz band among various RF-ID frequency bands such as 135kHz, 13.56MHz, 433MHz, 860~960MHz, 2.45GHz, etc. In addition, the UWB module can accurately measure ToA (Time of Arrival), which is the time it takes for a pulse to reach a target, and AoA (Ange of Arrival), which is the pulse arrival angle at the transmitting device, through communication between UWB antennas, and accordingly, precise distance and location recognition is possible within an error range of several tens of centimeters indoors.

[0178] In particular, in one or more embodiments, the processor (140) may receive information about the neural network model (1000), a learning data set of the neural network model (1000), information about the driving path of the robot cleaner (100), etc. from an external device through the communication unit (180). The processor (140) may receive a signal corresponding to a user input for controlling the operation of the robot cleaner (100) from the external device through the communication unit (180), and may receive a signal corresponding to a user input for modifying the driving path from the external device through the communication unit (180). The processor (140) may control the communication unit (180) to transmit information about the updated driving path to the external device. The processor (140) may control the communication unit (180) to transmit information corresponding to a message for guiding the user to detach or attach a wet brush to the external device.

[0179] The input unit (190) includes a circuit, and the processor (140) can receive a user command to control the operation of the robot cleaner (100) through the input unit (190). Specifically, the input unit (190) can be configured with components such as a microphone, a camera, and a remote control signal receiving unit. In addition, the input unit (190) can also be implemented in a form included in a display as a touch screen. In particular, the microphone can receive a voice signal and convert the received voice signal into an electrical signal.

[0180] In particular, in one or more embodiments, the processor (140) may receive user input, such as user input for controlling the operation of the robot cleaner (100), user input for modifying a driving path, etc. The processor (140) may receive user voice for controlling the operation of the robot cleaner (100) through a microphone.

[0181] The output unit (195) includes a circuit, and the processor (140) can output various functions that the robot cleaner (100) can perform through the output unit (195). In addition, the output unit (195) can include at least one of a display, a speaker, and an indicator.

[0182] The display can output image data under the control of the processor (140). Specifically, the display can output an image previously stored in the memory (130) under the control of the processor (140). In particular, the display according to one embodiment of the present disclosure can also display a user interface stored in the memory (130). The display can be implemented as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), etc., and in some cases, the display can also be implemented as a flexible display, a transparent display, etc. However, the display according to the present disclosure is not limited to a specific type.

[0183] The speaker can output audio data under the control of the processor (140), and the indicator can be turned on under the control of the processor (140).

[0184] In particular, in one or more embodiments, the processor (140) may control the output unit (195) to provide a message to inform the user that the driving path has been modified. The processor (140) may control the output unit (195) to provide a message to inform the user that the wet brush has been automatically raised or lowered. The processor (140) may control the output unit (195) to provide a message to inform the user to detach or attach the wet brush.

[0185] According to one or more embodiments described above with reference to FIGS. 8 and 9, the robot cleaner (100) can perform operations appropriate for the identified floor type. In particular, when the robot cleaner (100) enters a carpet area when it is identified that a carpet exists on the floor surface, the robot cleaner (100) can perform operations appropriate for the carpet area in the dry mode and the wet mode, respectively.

[0186] Meanwhile, FIGS. 2, 5, and 8 only describe the main components of the robot cleaner (100), and the robot cleaner (100) may further include components such as a suction port capable of sucking up contaminants, a camera capable of obtaining an image of a cleaning space, a GPS (Global Positioning System) sensor capable of obtaining information on the location of the robot cleaner (100), and a lidar sensor capable of obtaining information on the distance to an obstacle.

[0187] FIG. 10 is a flowchart illustrating a control method of a robot vacuum cleaner (100) according to one or more embodiments of the present disclosure.

[0188] Referring to FIG. 10, the robot cleaner (100) can acquire first data related to the characteristics of the floor surface along the driving path through the first sensor while the robot cleaner (100) moves along the driving path (S1010). Here, the first sensor is a sensor that uses ultrasonic waves, and the first data refers to data on the characteristics of the floor surface along the driving path of the robot cleaner (100) acquired through the first sensor.

[0189] Additionally, the robot cleaner (100) can acquire second data related to the movement of the robot cleaner (100) through at least one second sensor while the robot cleaner (100) moves along the driving path (S1020). The second sensor refers to a sensor related to the movement of the robot cleaner (100), and the second data refers to data related to the movement of the robot cleaner (100) acquired through at least one second sensor.

[0190] When the first data and the second data are acquired, the robot cleaner (100) inputs a driving data set including at least a portion of the first data and the second data into a trained neural network model (1000), thereby acquiring type information indicating the type of the floor surface (S1030). The neural network model (1000) refers to a model including a neural network trained to acquire information indicating the type of the floor surface based on the driving data set. The type information may include information on whether a carpet exists on the floor surface along the driving path, the material of the floor surface, and at least one of the material of the carpet.

[0191] Once the type information is acquired, the robot cleaner (100) can control the operation of the robot cleaner (100) based on the type information (S1040). Specifically, if it is determined that a carpet exists on the floor based on the type information, the robot cleaner (100) can be controlled to lift the wet brush while cleaning the carpet area where the carpet exists, and can also be controlled to bypass the carpet area by changing the driving path.

[0192] Meanwhile, the control method of the robot cleaner (100) according to the above-described embodiment may be implemented as a program and provided to the robot cleaner (100). In particular, the program including the control method of the robot cleaner (100) may be stored and provided in a non-transitory computer readable medium.

[0193] Specifically, in a non-transitory computer-readable recording medium including a program for executing a control method of a robot cleaner (100), the control method of the robot cleaner (100) includes a step of obtaining first data related to a feature of a floor surface on a driving path through a first sensor using ultrasonic waves while the robot cleaner (100) moves along a driving path, a step of obtaining second data related to a movement of the robot cleaner (100) through at least one second sensor related to the movement of the robot cleaner (100) while the robot cleaner (100) moves along the driving path, a step of inputting a driving data set including at least a portion of the first data and the second data into a learned neural network model (1000) to obtain type information indicating a type of a floor surface, and a step of controlling an operation of the robot cleaner (100) based on the type information.

[0194] In the above, the control method of the robot cleaner (100) and the computer-readable recording medium including the program for executing the control method of the robot cleaner (100) have been briefly described, but this is only to omit redundant description, and various embodiments of the robot cleaner (100) can of course also be applied to the control method of the robot cleaner (100) and the computer-readable recording medium including the program for executing the control method of the robot cleaner (100).

[0195] The artificial intelligence-related function according to the present disclosure is operated through the processor (140) and memory (130) of the robot cleaner (100). The processor (140) may be composed of one or more processors (140). In this case, the one or more processors (140) may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors (140) described above.

[0196] The CPU is a general-purpose processor (140) capable of performing not only general calculations but also artificial intelligence calculations. Its multi-layer cache structure allows for the efficient execution of complex programs. The CPU is advantageous in a serial processing method, which enables organic linking of previous and subsequent calculation results through sequential calculations. The general-purpose processor (140) is not limited to the aforementioned examples, except in cases where it is specifically designated as a CPU.

[0197] A GPU is a processor (140) for large-scale operations such as floating point operations used in graphic processing, and can perform large-scale operations in parallel by integrating a large number of cores. In particular, a GPU may be advantageous compared to a CPU in parallel processing methods such as convolution operations. In addition, a GPU may be used as a co-processor (140) to supplement the functions of a CPU. The processor (140) for large-scale operations is not limited to the examples described above, except in cases where it is specifically referred to as a GPU.

[0198] An NPU is a processor (140) specialized in artificial intelligence operations using an artificial neural network, and each layer constituting the artificial neural network can be implemented with hardware (e.g., silicon). At this time, since the NPU is designed specifically according to the required specifications of the company, it has a lower degree of freedom compared to a CPU or GPU, but it can efficiently process the artificial intelligence operations requested by the company. Meanwhile, as a processor (140) specialized in artificial intelligence operations, the NPU can be implemented in various forms such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), a Vision Processing Unit (VPU), etc. The artificial intelligence processor (140) is not limited to the above-described examples, except in cases where it is specified as the above-described NPU.

[0199] Additionally, one or more processors (140) may be implemented as a System on Chip (SoC). In this case, the SoC may further include, in addition to one or more processors (140), a memory (130), and a network interface such as a bus for data communication between the processor (140) and the memory (130).

[0200] When a plurality of processors (140) are included in a SoC (System on Chip) included in a robot cleaner (100), the robot cleaner (100) can perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the plurality of processors (140). For example, the robot cleaner (100) can perform operations related to artificial intelligence by using at least one of a GPU, an NPU, a VPU, a TPU, and a hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors (140). However, this is merely an example, and it is of course possible to process operations related to artificial intelligence by using a CPU or a general-purpose processor (140).

[0201] In addition, the robot cleaner (100) can perform calculations for functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in a single processor (140). In particular, the robot cleaner (100) can perform artificial intelligence calculations, such as convolution operations and matrix multiplication operations, in parallel by utilizing multiple cores included in the processor (140).

[0202] One or more processors (140) are controlled to process input data according to predefined operation rules or artificial intelligence models stored in the memory (130). The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0203] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0204] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0205] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0206] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0207] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smartphones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be at least temporarily stored or temporarily created in a device-readable storage medium, such as a manufacturer's server, an application store's server, or a memory (130) of an intermediary server.

[0208] Each of the components (e.g., modules or programs) according to the various embodiments of the present disclosure as described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in the various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration.

[0209] According to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0210] Meanwhile, the terms "part" or "module" used in the present disclosure include units composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A "part" or "module" may be an integrally composed component, a minimum unit performing one or more functions, or a portion thereof. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0211] Various embodiments of the present disclosure may be implemented as software including commands stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call commands stored in the storage medium and operate according to the called commands, and may include an electronic device (e.g., a robot vacuum cleaner (100)) according to the disclosed embodiments.

[0212] When the above instruction is executed by the processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter.

[0213] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In robot vacuum cleaners, A first sensor using ultrasound; At least one second sensor related to the movement of the robot cleaner; A memory for storing information on the driving path of the robot cleaner; While the robot cleaner moves along the driving path, first data related to the characteristics of the floor surface on the driving path is acquired through the first sensor, and second data related to the movement of the robot cleaner is acquired through the at least one second sensor. By inputting a driving data set including at least a portion of the first data and the second data into a learned neural network model, type information indicating the type of the floor surface is obtained, A robot cleaner comprising a processor that controls the operation of the robot cleaner based on the type information.

2. In paragraph 1, The at least one second sensor comprises at least one of an acceleration sensor, an angular velocity sensor and a wheel sensor, A robot cleaner, wherein the type information includes information on whether a carpet exists on the floor surface along the driving path, information on at least one of the material of the floor surface and the material of the carpet.

3. In paragraph 1, The above processor, By inputting only at least a part of the second data among the first data and the second data into the neural network model, the type information is obtained, Identify the type of the floor surface based on the first data, If it is identified that a carpet exists on the floor surface based on the first data, the operation of the robot cleaner is controlled based on the identification result that a carpet exists on the floor surface. A robot cleaner that controls the operation of the robot cleaner based on the type information when it is determined that no carpet exists on the floor surface based on the first data.

4. In paragraph 1, further comprising at least one third sensor utilizing infrared; The above processor, Obtaining third data related to the color of the floor surface and including information on the reflectivity of the floor surface for the infrared ray through the at least one third sensor, By inputting a driving data set including at least some of the first data, the second data and the third data into a learned neural network model, the type information is obtained, A robot cleaner, wherein the above driving data set includes data related to shaking of the robot cleaner among the second data and the third data.

5. In paragraph 4, The above processor, A robot cleaner including the third data in the driving data set when a value representing the amount of change in data related to shaking of the robot cleaner is greater than or equal to a first threshold value and a value representing the amount of change in the third data is greater than or equal to a second threshold value.

6. In paragraph 4, The above processor, A part of the driving data set is converted so that the larger the value representing the amount of change in data related to the shaking of the robot cleaner and the larger the value representing the amount of change in the third data, A robot vacuum cleaner that obtains the type information by inputting the above-mentioned converted driving data set into the above-mentioned neural network model.

7. In paragraph 1, dry brush; wet brush; A driving unit including at least one motor; further comprising: The above processor, A robot cleaner that controls the driving unit to lift the wet brush while performing cleaning on the carpet area where the carpet exists, when it is identified that a carpet exists on the floor surface based on the type information while performing cleaning on the floor surface using the wet brush.

8. In paragraph 7, The above processor, A robot cleaner that controls the driving unit to increase the suction power of the robot cleaner while cleaning the carpet area, if it is determined that a carpet exists on the floor surface based on the type information while the robot cleaner performs cleaning of the floor surface using the dry brush.

9. In paragraph 7, The above processor, Modify the information about the driving route stored in the memory based on the above type information, A robot cleaner that controls the driving unit to drive the robot cleaner based on information about the modified driving path.

10. In paragraph 7, The above processor, When cleaning the floor surface using the wet brush is performed, if it is identified that a carpet of the first material exists on the floor surface based on the type information, the driving unit is controlled to lift the wet brush while cleaning the carpet area where the carpet of the first material exists. A robot cleaner that, while performing cleaning on the floor using the wet brush, if it is identified that a carpet of a second material different from the first material exists on the floor based on the type information, modifies information about the driving path stored in the memory, and controls the driving unit so that the robot cleaner drives based on the information about the modified driving path.

11. In paragraph 7, The above wet brush includes a first wet brush arranged on a first side of the robot cleaner and a second wet brush arranged on a second side different from the first side, The above processor, A robot cleaner that controls the driving unit to lift the first wet brush and not to lift the second wet brush while performing cleaning on the composite area using the wet brush, when a composite area in which a carpet exists on a floor surface corresponding to the first side and no carpet exists on a floor surface corresponding to the second side is identified based on the type information.

12. In a method for controlling a robot vacuum cleaner, A step of obtaining first data related to the characteristics of a floor surface along the driving path through a first sensor using ultrasonic waves while the robot cleaner moves along the driving path; A step of acquiring second data related to the movement of the robot cleaner through at least one second sensor related to the movement of the robot cleaner while the robot cleaner moves along the driving path; A step of inputting a driving data set including at least a portion of the first data and the second data into a learned neural network model to obtain type information indicating the type of the floor surface; and A method for controlling a robot cleaner, comprising: a step of controlling the operation of the robot cleaner based on the type information; 13. In paragraph 12, The at least one second sensor comprises at least one of an acceleration sensor, an angular velocity sensor and a wheel sensor, A control method for a robot cleaner, wherein the type information includes information on whether a carpet exists on the floor surface along the driving path, information on at least one of the material of the floor surface and the material of the carpet.

14. In paragraph 12, The step of obtaining the type information includes a step of obtaining the type information by inputting only at least a part of the second data among the first data and the second data into the neural network model; The steps for controlling the operation of the above robot vacuum cleaner are: A step of identifying the type of the floor surface based on the first data; and If it is identified that a carpet exists on the floor surface based on the first data, a step of controlling the operation of the robot cleaner based on the identification result that a carpet exists on the floor surface; and A method for controlling a robot cleaner, further comprising: a step of controlling an operation of the robot cleaner based on the type information when it is determined that no carpet exists on the floor surface based on the first data; 15. In paragraph 12, The control method of the above robot vacuum cleaner is: A step of obtaining third data related to the color of the floor surface and including information on the reflectivity of the floor surface for the infrared ray through at least one third sensor using infrared ray; further comprising; The steps for obtaining the above type information are: A step of inputting a driving data set including at least some of the first data, the second data, and the third data into a learned neural network model to obtain the type information; including; A method for controlling a robot cleaner, wherein the driving data set includes data related to shaking of the robot cleaner among the second data and the third data.

Images