Robot device for identifying uneven area and control method therefor

Abstract

A robot device is disclosed. The robot device comprises: a memory; a first LiDAR sensor configured to detect obstacles in a first direction; and a second LiDAR sensor configured to detect obstacles in a second direction opposite to the first direction, wherein the robot device is configured to obtain floor information on a floor of a space in which the robot device is located on the basis of first sensing data received from the first LiDAR sensor and second sensing data received from the second LiDAR sensor, identify a plurality of subspaces included in the space on the basis of the floor information, obtain driving level information including a driving level related to each of the plurality of subspaces, store the driving level information in the memory, and control movement of the robot device on the basis of a driving level of a subspace corresponding to the position of the robot device among the plurality of subspaces obtained from the driving level information stored in the memory.

Description

Robotic device for identifying uneven areas and its control method

[0001] The present disclosure relates to a robot device and a control method thereof, and more particularly, to a robot device for identifying a flat area and an uneven area in a space and a control method thereof.

[0002] Recently, robotic devices are rapidly spreading in various fields, and not only are they replacing human roles, but unmanned factories (or smart factories) that operate solely with robotic devices without humans are also rapidly increasing.

[0003] As robotic devices move autonomously within a factory, they need to sense their surroundings with high accuracy to enable them to move more easily.

[0004] For example, there is a need to detect whether the floor on which the robot device is driving is flat or uneven (e.g., bumpy), and while the robot device is driving on an uneven floor, there is a need to drive safely to prevent an accident from occurring where an object loaded on the robot device is falling.

[0005] In the past, robot devices could only detect obstacles and drive to avoid them, but there was a problem in that it was difficult to adjust the driving speed of the robot device or drive to avoid them by considering the condition of the ground (e.g., whether there are uneven curves, etc.).

[0006] Accordingly, there have been various demands for methods for robotic devices located in factories and other places to detect the condition of the floor with greater accuracy and adjust the driving speed or driving path by taking the condition of the floor into consideration.

[0007] According to an embodiment of the present disclosure, a robot device comprises: a memory; a first LiDAR sensor configured to detect obstacles in a first direction; a second LiDAR sensor configured to detect obstacles in a second direction opposite to the first direction; and a second LiDAR sensor configured to obtain floor information about a floor of a space in which the robot device is located based on first sensing data received from the first LiDAR sensor and second sensing data received from the second LiDAR sensor; identify a plurality of sub-spaces included in the space based on the floor information; obtain driving level information including a driving level related to each of the plurality of sub-spaces; store the driving level information in the memory; and control movement of the robot device based on a driving level of a sub-space corresponding to a location of the robot device among the plurality of sub-spaces, obtained from the driving level information stored in the memory.

[0008] A method for controlling a robot device according to an embodiment of the present disclosure includes the steps of receiving first sensing data from a first lidar sensor in a first direction with respect to the robot device, receiving second sensing data from a second lidar sensor in a second direction opposite to the first direction, obtaining floor information about a floor of a space in which the robot device is located based on the first sensing data and the second sensing data, identifying a plurality of sub-spaces included in the space based on the floor information, obtaining driving level information including a driving level related to each of the plurality of sub-spaces, and controlling driving of the robot device based on a driving level of a sub-space corresponding to a location of the robot device among the plurality of sub-spaces.

[0009] According to an embodiment of the present disclosure, a non-transitory computer-readable recording medium including a program for executing a method for controlling a robot device includes the steps of: receiving first sensing data from a first lidar sensor in a first direction with respect to the robot device; receiving second sensing data from a second lidar sensor in a second direction opposite to the first direction; obtaining floor information about a floor of a space in which the robot device is located based on the first sensing data and the second sensing data; identifying a plurality of sub-spaces included in the space based on the floor information; obtaining driving level information including a driving level related to each of the plurality of sub-spaces; and controlling driving of the robot device based on a driving level of a sub-space corresponding to a location of the robot device among the plurality of sub-spaces.

[0010] According to an embodiment of the present disclosure, a device for controlling a robotic device includes at least one memory configured to store a program code, and at least one processor configured to read the program code and operate as directed by the program code. The program code includes a first receiving code that operates the at least one processor to receive first sensing data from a first lidar sensor in a first direction with respect to the robot device, a second receiving code that operates the at least one processor to receive second sensing data from a second lidar sensor in a second direction opposite to the first direction, a first acquisition code that operates the at least one processor to obtain floor information about a floor of a space in which the robot device is located based on the first sensing data and the second sensing data, a first identification code that operates the at least one processor to identify a plurality of sub-spaces included in the space based on the floor information, a second acquisition code that operates the at least one processor to obtain driving level information including a driving level associated with each of the plurality of sub-spaces, and a first control code that operates the at least one processor to control movement of the robot device based on a driving level of a sub-space corresponding to a location of the robot device among the plurality of sub-spaces.

[0011] FIG. 1 is a drawing for explaining the driving of a robot device according to one embodiment of the present disclosure.

[0012] FIG. 2 is a block diagram of a robot device according to an embodiment of the present disclosure.

[0013] FIG. 3 is a diagram illustrating views related to LiDAR sensors according to one embodiment of the present disclosure.

[0014] FIG. 4 is a diagram illustrating data of one or more lidar sensors according to one embodiment of the present disclosure.

[0015] FIG. 5 is a drawing illustrating a depth image according to one embodiment of the present disclosure.

[0016] FIG. 6A is a drawing illustrating a pitch change of a robot device according to an embodiment of the present disclosure.

[0017] FIG. 6b is a drawing illustrating a roll change of a robot device according to an embodiment of the present disclosure.

[0018] FIG. 7 is a drawing illustrating a driving and / or moving speed of a robot device according to one embodiment of the present disclosure.

[0019] FIG. 8 is a drawing for explaining an object loaded on a robot device according to an embodiment of the present disclosure.

[0020] FIG. 9 is a drawing illustrating evasive driving of a robot device according to an embodiment of the present disclosure.

[0021] FIG. 10 is a flowchart illustrating a control method of a robot device according to an embodiment of the present disclosure.

[0022] FIG. 11 is a flowchart illustrating a control method of a robot device according to an embodiment of the present disclosure.

[0023] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0024] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0025] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0026] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0027] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0028] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0029] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0030] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0031] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0032] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0033] FIG. 1 is a drawing for explaining the driving of a robot device according to one embodiment of the present disclosure.

[0034] As illustrated in FIG. 1, the robotic device (100) may refer to various types of devices capable of performing a task on its own. For example, the robotic device (100) may include a smart device that senses the surrounding environment of the robotic device (100) in real time based on sensing data from a sensor (e.g., a LiDAR (Light Detection And Ranging) sensor, a camera (e.g., a depth camera, an RGB camera, etc.) and operates autonomously by collecting information, in addition to a simple repeating function.

[0035] A robot device (100) according to one embodiment of the present disclosure may include a driving unit including an actuator or a motor. The driving unit according to one embodiment may include wheels, brakes, etc., and the robot device (100) may move within a space on its own using the wheels, brakes, etc. included in the driving unit.

[0036] The robotic device (100) according to the embodiment may include a robotic joint (articulation or joint). In the present disclosure, a robotic joint may refer to a component of the robotic device (100) that replaces the function of a human arm or hand.

[0037] A robot device (100) according to an embodiment of the present disclosure is equipped with a sensor, and can obtain map information corresponding to a space (e.g., a preset space in a factory, a preset space in a home) in which the robot device (100) is located based on sensing data of the sensor.

[0038] For example, the robot device (100) is equipped with a LiDAR sensor and can generate a 2D or 3D map corresponding to space using sensing data of the LiDAR sensor and various mapping algorithms.

[0039] A LiDAR (Light Detection And Ranging) sensor is a sensor that measures the distance to a specific object. Specifically, a LiDAR sensor can perform sensing by emitting light using a light source, and when the light is emitted (or released) by a target object, detecting the reflected light using a sensor around the light source. A robot device (100) can measure the time it takes for the light to return using a LiDAR sensor, and calculate the distance to the target based on the measured time and the speed of light. The robot device (100) can generate a 3D (or 2D) map for a specific space by periodically repeating measurements using a LiDAR sensor. Here, the map can include a 2D or 3D image or information indicating the position, size, shape, etc. of the entire space and a plurality of sub-spaces included in the space. The map may further include 2D or 3D images or information indicating the location, size, shape, etc. of obstacles (e.g., factory equipment (e.g., production machinery), furniture, appliances, etc.) contained in the entire space and each of multiple sub-spaces. The map information may be various types of information or data used to construct such a map.

[0040] The robot device (100) according to the embodiment can identify the location of the robot device (100) within the map based on sensing data.

[0041] For example, the robot device (100) can identify a movement path based on map information, and can identify the location of the robot device (100) within the map while the robot device (100) moves within the space according to the movement path.

[0042] For example, the robot device (100) can perform SLAM (simultaneous localization and mapping) operation to obtain map information corresponding to the space where the robot device (100) is located, and identify the current location of the robot device (100) within the map.

[0043] Meanwhile, the robot device (100) may be classified into industrial, medical, household, military, and exploration robots depending on the functions it can perform. According to one embodiment, the industrial robot device may be further classified into a robot device used in the product manufacturing process in a factory, a robot device that performs customer service, order reception, and serving in a store or restaurant, etc. For example, the robot device (100) may be implemented as a serving robot device that can transport service items to a specific location desired by a user in various places such as a restaurant, hotel, supermarket, hospital, or clothing store.

[0044] However, this is not limited thereto, and the robot device (100) can be classified into various categories depending on the field of application, function, and purpose of use. For example, the robot device (100) can be implemented in various forms such as a mobile projector with a projection function, a robot vacuum cleaner with a cleaning function, and a service robot for delivering or transferring various items. Depending on the form of implementation, the robot device (100) can further include various components such as a projection unit (not shown), a dry cleaning module, a wet cleaning module, a robot arm, and the like, but detailed drawings and descriptions thereof are omitted.

[0045] According to an example of the present disclosure, the robotic device (100) can obtain floor information of a space based on sensing data. In one example, the robotic device (100) can divide the space into a plurality of sub-spaces based on the floor information.

[0046] For example, floor information may include information related to flat areas and information related to uneven areas within a space.

[0047] For example, a flat area may be an area where the uniformity of the floor surface is within a certain acceptable range. The flat area may be referred to as a low-risk area, an area where the robot device (100) can easily move, a less dangerous area, etc., but will be collectively referred to as a flat area hereinafter.

[0048] For example, an uneven area may be an area where the uniformity of the floor surface exceeds an allowable range, i.e., an area where the floor is uneven beyond the tolerable range. An uneven area may be called a high-risk area, an area where the movement of the robot device (100) is not easy, a dangerous area, or an uneven area, but hereinafter, it will be collectively referred to as an uneven area.

[0049] For example, information related to a flat area may include the location, size (or, the surface), shape, etc. of a flat area within a space, and information related to an uneven area may include the location, size, shape, etc. of an uneven area within a space.

[0050] For example, the robot device (100) can obtain map information that divides a space into multiple sub-spaces based on floor information.

[0051] For example, the robot device (100) can control the movement of the robot device (100) by identifying whether the sub-space in which the robot device (100) is to move among a plurality of sub-spaces is a flat area or an uneven area based on map information.

[0052] According to an embodiment, the robot device (100) may divide the space into multiple sub-spaces by identifying flat and uneven areas within the space, or may divide the space into multiple sub-spaces by identifying independent areas surrounded by walls. This is an example, and the robot device (100) may also divide the space into multiple sub-spaces by identifying furniture, stairs, thresholds, etc. within the space.

[0053] FIG. 2 is a block diagram illustrating a robot device according to an embodiment of the present disclosure.

[0054] Referring to FIG. 2, the robot device (100) includes a sensor (110), a driving unit (120), and a main module (130).

[0055] According to an embodiment, the sensor (110) may include a plurality of lidar sensors, such as a first lidar sensor (111) and a second lidar sensor (112). The sensor (110) may also include a camera (113). The main module (130) may include a communication interface (131), a memory (132) (including one or more memory chips and types), at least one processor (133), and a control unit (134).

[0056] The sensor (110) is configured to sense various information. According to an embodiment, at least one processor (133) may acquire various information based on the sensing values ​​of the sensor (110). For example, the information acquired by the sensor (110) may include image and depth information. The image may include RGB values ​​of each of a plurality of pixels included in the image. The depth information may include a depth map including depth values ​​of each of a plurality of pixels.

[0057] According to an embodiment, the sensor (110) includes a first lidar sensor (111) and a second lidar sensor (112).

[0058] Each of the first lidar sensor (111) and the second lidar sensor (112) can emit light toward a target object using a light source and detect light reflected from the target object. At least one processor (133) can identify the distance to the target object based on the time taken for the light to be reflected from the target object and detected after the light is emitted from each of the first lidar sensor (111) and the second lidar sensor (112).

[0059] According to an embodiment of the present disclosure, the first lidar sensor (111) may be configured to detect a first direction (or emit light in the first direction) with respect to the robot device (100). In the present disclosure, the first direction may be the driving direction of the robot device (100) when the robot device (100) is moving forward, or a direction toward the front of the robot device (100).

[0060] For example, the first lidar sensor (111) can detect the distance to an obstacle located in front of the robot device (100) when the robot device (100) moves forward. However, this is an example, and the first direction may correspond to a side (e.g., the right side, the left side, etc.) of the robot device (100).

[0061] According to an embodiment of the present disclosure, the second lidar sensor (112) may be provided to detect a second direction opposite to the first direction with respect to the robot device (100) (or to emit light in the second direction). In the present disclosure, the second direction may be a direction opposite to the driving direction of the robot device (100) when the robot device (100) is moving forward, or a direction corresponding to the rear of the robot device (100).

[0062] For example, the second lidar sensor (112) can detect the distance to an obstacle located at the rear of the robot device (100) when the robot device (100) moves forward. However, this is an example, and the second direction may correspond to a side of the robot device (100) (e.g., the right side, the left side, etc.). In addition, the first direction and the second direction may be implemented as opposite directions to each other, but are not necessarily limited thereto, and may be implemented as two different directions related by a certain angle or more.

[0063] According to an embodiment, the first lidar sensor (111) may obtain first sensing data detecting a distance to an obstacle located in a first direction, and the second lidar sensor (112) may obtain second sensing data detecting a distance to an obstacle located in a second direction.

[0064] According to an example, the sensor (110) may include a camera (113). The camera (113) may include a stereo camera, an RGB-D camera, a ToF (Time of Flight) camera, a depth camera, etc. However, the present invention is not limited to this example, and the sensor (110) may include various sensors capable of acquiring images and depth information.

[0065] The driving unit (120) can control the movement of the robot device (100). For example, the driving unit (120) can control the movement of the robot device (100), stop the moving robot device (100), and / or control the moving speed and / or moving direction of the robot device (100).

[0066] In embodiments, the robot device (100) can move using an appropriate drive device, such as a wheel type drive device, a walking type drive device, etc.

[0067] The wheel type refers to the way the robot device (100) moves by rotating the wheels. If the robot device (100) is a wheel-type robot, the robot device (100) may include one or more wheels. The driving unit (120) may include a device that generates power to rotate the wheels. For example, the driving unit (120) may be implemented as a gasoline engine, a diesel engine, an LPG (liquefied petroleum gas) engine, or an electric motor, depending on the fuel (or energy source) used.

[0068] The walking type refers to the way in which the robot device (100) moves through the movement of the legs. If the robot device (100) is a walking type (e.g., a bipedal walking robot, a triped walking robot, a quadruped walking robot, etc.), the robot device (100) may include two or more legs that support the robot device (100). The legs may include a plurality of links and joints connected to the links. The driving unit (120) may include a device that generates power to raise or lower the legs by rotating the links around the joints. For example, the driving unit (120) may be implemented with a motor and / or an actuator.

[0069] Additionally, the driving unit (120) can control the movement of a part of the robot device (100). The driving unit (120) can be coupled between a first part (e.g., a body) and a second part (e.g., a head, an arm, etc.) of the robot device (100). The driving unit (120) can rotate the second part. For example, the driving unit (120) can be implemented with a motor and / or an actuator, etc.

[0070] The main module (130) is implemented in hardware and may include a communication interface (131), memory (132), at least one processor (133), and a control unit (134).

[0071] The communication interface (131) can perform data communication with electronic devices under the control of at least one processor (133). For example, the communication interface (131) can include a communication circuit that can perform data communication between the robot device (100) and the electronic device by using at least one of data communication methods including wired LAN, wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliances (WiGig), and RF communication.

[0072] Memory (132) may store instructions, data structures, and program codes that can be read by at least one processor (133). Operations performed by at least one processor (133) may be implemented by executing instructions or codes of a program stored in memory (132).

[0073] The memory (132) may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0074] According to an embodiment, at least one processor (133) controls the overall operation of the electronic device (100). Specifically, at least one processor (133) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100).

[0075] At least one processor (133) can perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (132).

[0076] At least one processor (133) may include one or more of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), an MIC (Many Integrated Core), a DSP (Digital Signal Processor), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. At least one processor (133) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. At least one processor (133) may execute one or more programs or instructions stored in the memory (132). For example, at least one processor (133) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory (132).

[0077] If a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0078] At least one processor (133) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When at least one processor (133) is implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0079] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0080] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) having at least one processor and other electronic components integrated therein, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0081] According to an embodiment of the present disclosure, at least one processor (133) can obtain floor information of a space where a robot device (100) is located based on first sensing data received from a first lidar sensor (111) and second sensing data received from a second lidar sensor (112).

[0082] At least one processor (133) according to an embodiment can identify a plurality of sub-spaces included in a space based on floor information.

[0083] The control unit (134) can control components of the robot device (100). The control unit (134) can control components of the robot device (100) (e.g., sensor (110) and driving unit (120), etc.) based on signals provided from at least one processor (133). For example, the control unit (134) can generate a control signal using signals provided from at least one processor (133) and provide the control signal to components of the robot device (100). Accordingly, the components of the robot device (100) can perform operations corresponding to the operation results of at least one processor (133). The control unit (134) can be implemented with one or more ICs (e.g., controller ICs).

[0084] FIG. 3 is a drawing illustrating a LiDAR sensor according to one embodiment of the present disclosure.

[0085] According to at least one embodiment, while the robot device (100) is driving under the control of at least one processor (133), the at least one processor (133) can obtain map information of the space in which the robot device (100) is located, for example, according to a suitable method, for example, a SLAM operation, and identify the current location of the robot device (100) within the space.

[0086] Referring to scenario 300 of FIG. 3, when the robot device (100) drives on a flat area, light emitted by the first lidar sensor (111) is reflected by an obstacle (e.g., a wall, etc.) located in a first direction based on the robot device (100), and the light reflected by the obstacle can be received by the first lidar sensor (111).

[0087] According to an embodiment, at least one processor (133) may identify a first distance to an obstacle located in front of the robot device (100) based on first sensing data received from the first lidar sensor (111).

[0088] The light emitted by the second lidar sensor (112) is reflected by an obstacle located in the second direction based on the robot device (100), and the light reflected by the obstacle can be received by the second lidar sensor (112).

[0089] According to an embodiment, at least one processor (133) may identify a second distance to an obstacle located at the rear of the robot device (100) based on second sensing data received from the second lidar sensor (112).

[0090] According to an embodiment, when the robot device (100) drives on a flat area, the first distance from an obstacle detected through the first lidar sensor (111) may be constantly reduced, and the second distance from an obstacle detected through the second lidar sensor (112) may be constantly increased.

[0091] According to an embodiment, when an obstacle is positioned in front of a robot device (100) and the robot device (100) moves in a first direction, the distance between the robot device (100) and the obstacle decreases in response to the moving speed of the robot device (100), so that the first distance to the obstacle detected through the first lidar sensor (111) can be constantly decreased.

[0092] In addition, when an obstacle is located at the rear of the robot device (100) and the robot device (100) moves in the first direction, the distance between the robot device (100) and the obstacle increases in response to the moving speed of the robot device (100), so that the second distance to the obstacle detected through the second lidar sensor (112) can increase constantly.

[0093] According to an embodiment, at least one processor (133) may identify an area in which the robot device (100) has moved as a flat area, based on the first sensing data received from the first lidar sensor (111) while the robot device (100) is driving, if a first distance to an obstacle located in front of the robot device (100) is constantly reduced (e.g., corresponding to a moving speed of the robot device (100)) or a second distance to an obstacle located in the rear of the robot device (100) is constantly increased based on the second sensing data received from the second lidar sensor (112).

[0094] At least one processor (133) may acquire flat area related information including at least one of a location, size, or shape of an area identified as a flat area while acquiring map information.

[0095] Referring to scenarios 310 and 320 of FIG. 3, at least one processor (133) can identify an area in which the robot device (100) has moved as an uneven area if at least one of the first distance to the obstacle detected through the first lidar sensor (111) or the second distance to the obstacle detected through the second lidar sensor (112) rapidly increases and decreases.

[0096] As illustrated in scenario 310 of FIG. 3, when the robot device (100) is driving on an uneven area and the robot device (100) tilts forward, the light emitted by the first lidar sensor (111) is reflected on the floor and then detected, so that the first distance from the obstacle (e.g., the floor) identified based on the first sensing data received from the first lidar sensor (111) may be rapidly reduced.

[0097] For example, rather than the change in the first distance to an obstacle located in front of the robot device (100) decreasing in response to the moving speed of the robot device (100), the change in the first distance may exceed the moving speed of the robot device (100).

[0098] For example, while the robot device (100) is driving on a flat area, the light emitted by the first lidar sensor (111) is reflected on the wall located in front of the robot device (100) (e.g., scenario 300), and while the robot device (100) is driving on an uneven area, when the robot device (100) leans forward, the light emitted by the first lidar sensor (111) is reflected on the floor located in front of the robot device (100) (e.g., scenario 310), so that the first distance to the obstacle identified based on the first sensing data received from the first lidar sensor (111) can be rapidly reduced.

[0099] As illustrated in scenario 320 of FIG. 3, when the robot device (100) is driving on an uneven area and the robot device (100) tilts backward, the light emitted by the second lidar sensor (112) is reflected on the floor and then detected, so that the second distance from the obstacle (e.g., the floor) identified based on the second sensing data received from the second lidar sensor (112) may be rapidly reduced.

[0100] For example, the amount of change in the second distance to an obstacle located at the rear of the robot device (100) may not increase in response to the movement speed of the robot device (100), but the amount of change in the second distance may exceed the movement speed of the robot device (100).

[0101] For example, while the robot device (100) is driving on a flat area, the light emitted by the second lidar sensor (112) is reflected on the wall located at the rear of the robot device (100), and while the robot device (100) is driving on an uneven area, when the robot device (100) tilts backward, the light emitted by the second lidar sensor (112) is reflected on the floor located at the rear of the robot device (100), so that the second distance to the obstacle identified based on the second sensing data received from the second lidar sensor (112) can be rapidly reduced.

[0102] According to an embodiment, at least one processor (133) may identify an area where the robot device (100) is located as an uneven area when a sudden increase or sudden decrease in a first distance is identified based on first sensing data received from a first lidar sensor (111) while the robot device (100) is driving, or a sudden increase or sudden decrease in a second distance is identified based on second sensing data received from a second lidar sensor (112).

[0103] At least one processor (133) may acquire non-flat area related information including at least one of the location, size, or shape of an area identified as a non-flat area while acquiring map information.

[0104] According to an embodiment, at least one processor (133) may predict the position of the robot device (100) at a time when an uneven area is identified. Then, at least one processor (133) may adjust the predicted position to an actual position of the robot device (100) based on first sensing data received from the first lidar sensor (111), second sensing data received from the second lidar sensor (112), and images received from the camera (113). Then, at least one processor (133) may map an area corresponding to the adjusted position on the map as an uneven area.

[0105] According to an embodiment, at least one processor (133) may, while acquiring map information of a space in which a robot device (100) is located according to a SLAM operation, identify (or, segment, extract) an area mapped as an uneven area, thereby identifying at least one of the position, size, or shape of the uneven area within the space.

[0106] Additionally, at least one processor (133) can predict the position of the robot device (100) at the time when the flat area is identified. Then, at least one processor (133) can adjust the predicted position to the actual position of the robot device (100) based on the first sensing data received from the first lidar sensor (111), the second sensing data received from the second lidar sensor (112), and the image received from the camera (113). Then, at least one processor (133) can map an area corresponding to the adjusted position on the map as a flat area.

[0107] According to an embodiment, at least one processor (133) may, while acquiring map information of a space in which a robot device (100) is located according to a SLAM operation, identify (or, segment, extract) an area mapped as a flat area, thereby identifying at least one of the position, size, or shape of the flat area within the space.

[0108] FIG. 4 is a diagram illustrating first sensing data and second sensing data according to an embodiment of the present disclosure.

[0109] Referring to the time interval t0-t1 of scenario 300 of FIG. 3 and diagram 400 of FIG. 4, when the robot device (100) moves on a flat area, the first distance (A) with the wall located in front of the robot device (100) may decrease corresponding to the moving speed of the robot device (100).

[0110] In addition, as shown in the time section t0-t1 of diagram 450 of FIG. 4, when the robot device (100) moves on a flat area, the second distance (B) with the wall located at the rear of the robot device (100) may increase corresponding to the moving speed of the robot device (100).

[0111] According to an embodiment, at least one processor (133) may identify an area in which the robot device (100) has moved as a flat area if the first distance (A) decreases corresponding to the moving speed of the robot device (100), or if the second distance (B) increases corresponding to the moving speed of the robot device (100).

[0112] According to an embodiment, if there is no obstacle positioned in front of the robot device (100), at least one processor (133) may identify the first distance (A) as '0' or infinity. For example, if there is no obstacle positioned in front of the robot device (100), the light emitted by the first lidar sensor (111) is not reflected by the obstacle, and therefore, if the light emitted by the first lidar sensor (111) does not return for a threshold time or longer, the at least one processor (133) may identify the first distance (A) as '0' or infinity. According to an embodiment, if the at least one processor (133) identifies the first distance (A) as '0' or infinity while the robot device (100) is driving, the area in which the robot device (100) has moved may be identified as a flat area.

[0113] According to an embodiment, at least one processor (133) may identify the second distance (B) as '0' or infinity if there is no obstacle located at the rear of the robot device (100). According to an embodiment, at least one processor (133) may identify the area in which the robot device (100) has moved as a flat area if the second distance (B) is identified as '0' or infinity while the robot device (100) is driving.

[0114] According to an embodiment, if an obstacle is not detected within the maximum distance detectable by the lidar sensor (110), for example, in the specifications of the lidar sensor (110), the area where the robot device (100) is located may not be identified as a flat area, but may be identified as a flat area or an uneven area based on sensing data from other sensors (e.g., an inertial measurement unit (IMU), a camera (113)) other than the lidar sensor (110).

[0115] Accordingly, it goes without saying that at least one processor (133) can identify an uneven area based on sensing data received from at least one sensor among a plurality of sensors, such as a lidar sensor (110), a camera (113), and an inertial measurement unit (IMU).

[0116] Referring to the scenario 310 of FIG. 3 and the time interval t1-t2 of the diagram 400 of FIG. 4, when the robot device (100) drives on an uneven area, particularly when the robot device (100) tilts forward, the first distance (A) from an obstacle (e.g., a floor) located in front of the robot device (100) may rapidly decrease. Accordingly, as illustrated in the time interval t1-t2 of the diagram 450 of FIG. 4, the second distance (B) from an obstacle (e.g., a wall) located at the rear of the robot device (100) may rapidly increase.

[0117] Referring to the time interval t1-t2 of scenario 320 of FIG. 3 and diagram 400 of FIG. 4, when a robot device (100) driving on an uneven area tilts backward, the obstacle located in front of the robot device (100) changes from the floor to a wall, so that the first distance (A) to the obstacle may rapidly increase. In addition, since the obstacle located at the rear of the robot device (100) changes from the wall to the floor, so that the second distance (B) to the obstacle may rapidly decrease.

[0118] According to an embodiment, when a rapid increase or decrease in the distance to an obstacle (e.g., the first distance (A) or the second distance (B)) is repeatedly detected based on at least one of the first sensing data or the second sensing data, as illustrated in the time interval t1-t2 of diagram 400 and diagram 450 of FIG. 4, at least one processor (133) may identify the area in which the robot device (100) has moved as an uneven area.

[0119] In various embodiments of the present disclosure, a rapid decrease and a rapid increase may include a change in distance (a decrease in distance, an increase in distance) that exceeds a unit time times the moving speed of the robotic device (100).

[0120] For example, at least one processor (133) may identify an area in which the robot device (100) has moved as an uneven area when the increase and decrease of the first distance is repeated for a preset time (e.g., 10 sec), or the increase and decrease of the second distance is repeated, and the amount of change in the distance exceeds the preset time multiplied by the moving speed of the robot device (100).

[0121] Here, the preset time may be inversely proportional to the movement speed of the robot device (100). For example, the faster the movement speed of the robot device (100) moving through an uneven area, the faster the increase and decrease in the distance between the lidar sensor (110) and the obstacle is repeated, so it is possible to identify whether the area through which the robot device (100) moved is an uneven area by identifying whether the distance increases or decreases in a relatively short period of time.

[0122] For example, the slower the moving speed of a robot device (100) moving on an uneven area, the slower the increase and decrease in the distance between the lidar sensor (110) and the floor is repeated, so it is possible to identify whether the area where the robot device (100) is located is an uneven area by identifying whether the distance increases and decreases over a relatively long period of time.

[0123] Accordingly, as shown in the time section t1-t2 of diagram 400 and diagram 450 of FIG. 4, at least one processor (133) may identify an area where the robot device (100) is located as an uneven area in the time section t1-t2 if the increase / decrease pattern or change (or change trend of the distance) of the distance from the floor detected in the first direction does not correspond to the moving speed of the robot device (100), or if the increase / decrease pattern or change of the distance from the floor detected in the second direction does not correspond to the moving speed of the robot device.

[0124] In the embodiment, it is assumed that at least one processor (133) identifies the area where the robot device (100) is located as an uneven area based on distance changes (e.g., distance increase / decrease pattern or fluctuation, distance change trend, etc.). However, this is not limited thereto, and at least one processor (133) may also identify the area where the robot device (100) is located as an uneven area by taking into account changes in the distance to an obstacle.

[0125] For example, as shown in the time interval t1-t2 of diagram 400 and diagram 450 of FIG. 4, at least one processor (133) may identify an area where the robot device (100) is located as an uneven area in the time interval t1-t2 if the pattern of increase or decrease or variation (or trend of change in distance) of the distance to an obstacle (e.g., a wall or equipment, etc.) detected in the first direction does not increase or decrease correspondingly to the moving speed of the robot device (100) (or, constantly according to the movement of the robot device (100)), or if the pattern of increase or decrease or variation of the distance to an obstacle detected in the second direction does not increase or decrease correspondingly to the moving speed of the robot device (100).

[0126] According to an embodiment, at least one processor (133) may identify an area in which the robot device (100) has moved as a flat area or an uneven area based on at least one of the first sensing data or the second sensing data, and may obtain floor information dividing the space into a flat area and an uneven area.

[0127] According to an embodiment, the floor information includes a plurality of sub-spaces, and each of the plurality of sub-spaces may correspond to a flat area and an uneven area.

[0128] FIG. 5 is a drawing illustrating a depth image according to one embodiment of the present disclosure.

[0129] The robotic device (100) according to the embodiment may include a depth camera.

[0130] The depth camera may include a Time of Flight (ToF) camera sensor.

[0131] The ToF camera sensor can irradiate a signal (e.g., near-infrared, ultrasound, laser, etc.) and, when the irradiated signal is reflected by a subject, receive the reflected signal. At least one processor (133) can measure the time until the ToF camera sensor receives the reflected signal, thereby measuring the distance (or depth) between the ToF camera sensor and the subject. In one example, at least one processor (133) can obtain depth information of the subject based on the distance between the ToF camera sensor and the subject.

[0132] The ToF camera sensor is an example, and is not limited thereto, and the robot device (100) may of course also include a radar sensor, an ultrasonic sensor, an infrared sensor, etc.

[0133] According to an embodiment, at least one processor (133) may control a depth camera to photograph the floor corresponding to the position of the robotic device (100).

[0134] For example, the depth camera can project a signal in a first direction corresponding to the front of the robot device (100) and receive a signal reflected from a subject (e.g., the floor).

[0135] According to an embodiment, at least one processor (133) can control the depth camera to capture a subject at preset time intervals to obtain multiple depth images.

[0136] According to an embodiment, each of the plurality of depth images can obtain a synthetic depth image that represents the height difference of the subject.

[0137] For example, at least one processor (133) can identify an uneven area (i.e., a non-flat area) due to a difference in height of the floor based on a plurality of depth images.

[0138] For example, at least one processor (133) obtains a synthetic depth image based on a plurality of depth images, and the synthetic depth image may be a 3D image that represents the difference in height (or change in height) of the floor by 3D modeling the floor.

[0139] According to an embodiment, at least one processor (133) can identify a plurality of sub-spaces by dividing the space into a flat area and an uneven area based on the first sensing data of the first lidar sensor (111) and the second sensing data of the second lidar sensor (112).

[0140] For example, at least one processor (133) can identify an independent area surrounded by a wall within a space to divide the space into a plurality of sub-spaces, and can also identify furniture, stairs, thresholds, etc. within the space to divide the space into a plurality of sub-spaces.

[0141] According to an embodiment, at least one processor (133) may divide each of the plurality of sub-spaces into a flat area and an uneven area. This is not limited thereto, and at least one processor (133) may divide a portion of a sub-space into an uneven area and the remainder into a flat area.

[0142] According to an embodiment, at least one processor (133) may identify a change in the height of the floor of a sub-space (or a part of the sub-space) corresponding to an uneven area among a plurality of sub-spaces based on a plurality of depth images received from a depth camera. According to an embodiment, at least one processor (133) may control the movement of the robot device (100) based on the identified change in height.

[0143] According to an embodiment, at least one processor (133) may identify a space into a plurality of sub-spaces based on a plurality of depth images received from a depth camera.

[0144] For example, at least one processor (133) may acquire a 3D image representing a change in height of a floor in a space while acquiring map information, acquire information related to an uneven area representing an area where a change in height is detected (or an area where a change in height is not included in a threshold range) as an uneven area, and acquire information related to a flat area representing an area where a change in height is not detected (or an area where a change in height is included in a threshold range) as a flat area.

[0145] As described above, at least one processor (133) can obtain floor information including information related to a flat area and information related to a non-flat area within the space.

[0146] FIG. 6A is a drawing illustrating a pitch change of a robot device based on an inertial measurement unit according to an embodiment of the present disclosure.

[0147] The robot device (100) according to the embodiment may include an inertial measurement unit (IMU).

[0148] An inertial measurement unit (hereinafter, IMU) according to an embodiment may include at least one of a gyroscope sensor, an acceleration sensor, or a magnetometer or compass sensor. Based on sensing values ​​of various inertial measurement devices, the robot device (100) may identify whether the robot device (100) is tilted, the direction of the tilt, the degree of the tilt, etc.

[0149] According to an embodiment, the IMU may obtain roll, pitch, and yaw, which represent the pose of the robot device (100), as third sensing data. Here, the roll may represent the left-right inclination of the robot device (100) (e.g., a rotation angle based on the x-axis (vertical axis)), the pitch may represent the forward-backward inclination of the robot device (100) (e.g., a rotation angle based on the y-axis (horizontal axis)), and the yaw may represent the z-axis inclination of the robot device (100) (e.g., a rotation angle based on the z-axis (vertical axis)).

[0150] According to an embodiment, at least one processor (133) may detect a pose of the robot device (100) based on third sensing data received from the IMU.

[0151] FIG. 6a is a side view of the robot device (100), and while the robot device (100) moves on an uneven area, the pitch of the robot device (100) can change.

[0152] For example, when the robot device (100) tilts forward, the pitch according to the third sensing data received from the IMU may be a positive (+) value, and when the robot device (100) tilts backward, the pitch according to the third sensing data received from the IMU may be a negative (-) value. It should be understood that the present disclosure is not limited to this embodiment.

[0153] At least one processor (133) according to an embodiment detects a pose of the robot device (100) based on the third sensing data, and when the detected pose exceeds a threshold angle (for example, when the pitch of the robot device (100) exceeds a threshold value), the pose of the robot device (100) can be tilted forward or backward, and the area in which the robot device (100) has moved can be identified as an uneven area.

[0154] FIG. 6b is a drawing illustrating a roll change of a robot device based on an inertial measurement device according to an embodiment of the present disclosure.

[0155] The drawing on the upper left of FIG. 6b includes a plan view showing the robot device (100) and the wall from above.

[0156] While the robot device (100) moves across an uneven area, the roll of the robot device (100) may change.

[0157] For example, if the robot device (100) tilts to the right, the roll according to the third sensing data received from the IMU may be a positive (+) value, and if the robot device (100) tilts to the left, the roll according to the third sensing data received from the IMU may be a negative (-) value. It should be understood that the present disclosure is not limited to this embodiment.

[0158] At least one processor (133) according to an embodiment detects a pose of the robot device (100) based on the third sensing data, and when the detected pose exceeds a threshold angle (for example, when the roll of the robot device (100) exceeds a threshold value), the pose of the robot device (100) can be tilted to the right or left, and the area in which the robot device (100) has moved can be identified as an uneven area.

[0159] According to an embodiment, at least one processor (133) may identify an area in which the robot device (100) has moved as a flat area or an uneven area based on at least one of the sensing data of the lidar sensor (110) (e.g., the first sensing data and the second sensing data) or the third sensing data of the IMU.

[0160] FIG. 7 is a drawing for explaining a driving level according to one embodiment of the present disclosure.

[0161] At least one processor (133) according to an embodiment can control the robot device (100) to drive at any one of a plurality of driving levels.

[0162] For example, multiple driving levels may be distinguished based on the driving speed of the robot device (100). Each of the multiple driving levels according to an embodiment of the present disclosure may include information for controlling the driving of the robot device (100).

[0163] For example, information for controlling driving may include a driving speed range of the robot device (100) (e.g., minimum speed and maximum speed), an average driving speed of the robot device (100), etc.

[0164] For example, among the plurality of driving levels, a first driving level may include high-speed driving of the robot device (100), a second driving level may include normal driving of the robot device (100), a third driving level may include low-speed driving of the robot device (100), and a fourth driving level may include evasive driving of the robot device (100). However, this is an example for convenience of explanation and is not limited thereto. For example, the first driving level may include high-speed driving of the robot device (100), and the second driving level may include low-speed driving of the robot device (100).

[0165] However, the present invention is not limited thereto, and information for controlling driving may include an operation mode of the robot device (100). For example, among the plurality of driving levels, a first driving level may include a power mode (e.g., a high output mode) of the robot device (100), a second driving level may include a normal mode (e.g., an eco mode) of the robot device (100), and a third driving level may include a low power mode (e.g., an eco mode) of the robot device (100). According to an embodiment, each of the plurality of driving levels may include information for controlling a speed, a turning radius, and an output (rotational speed) of a motor provided in the robot device (100) related to the operation of a gripper (e.g., a robot hand, which is an end portion of a multi-joint robot) provided in the robot device (100).

[0166] At least one processor (133) according to an embodiment may operate (or drive) the robot device (100) in one of a plurality of driving levels based on sensing data (e.g., first sensing data, second sensing data, depth image, third sensing data, etc.) received from a sensor (e.g., lidar sensor (110), depth camera, IMU, etc.) provided in the robot device (100). For example, at least one processor (133) may identify a driving level corresponding to the current location of the robot device (100) among a plurality of driving levels based on sensing data received in real time from the sensor.

[0167] According to an embodiment, at least one processor (133) can obtain a driving path (or movement path) of the robot device (100) by identifying a driving level corresponding to each of a plurality of sub-spaces within the space based on map information.

[0168] For example, at least one processor (133) may identify, based on map information, if a first sub-space among a plurality of sub-spaces corresponds to a flat area, the driving level of the first sub-space among the plurality of driving levels may be identified as the first driving level, and if a second sub-space corresponds to an uneven area, the driving level of the second sub-space among the plurality of driving levels may be identified as the second driving level. According to an embodiment, the driving speed of the robot device (100) according to the second driving level may be relatively slower than the driving speed according to the first driving level.

[0169] According to an embodiment, at least one processor (133) may obtain a driving path in which the robot device (100) drives at a first driving level while moving in a first sub-space, and in which the robot device (100) drives at a second driving level while moving in a second sub-space.

[0170] For example, when the robot device (100) drives in an uneven area, there is a risk of danger to objects (e.g., people, obstacles, etc.) adjacent to the robot device (100), so at least one processor (133) may operate the robot device (100) at a driving level (e.g., low-speed driving) corresponding to the uneven area based on map information while the robot device (100) drives in the uneven area.

[0171] However, the present invention is not limited thereto, and at least one processor (133) may identify a driving level of a second sub-space among multiple driving levels as an avoidance driving level.

[0172] For example, at least one processor (133) may control the robot device (100) to not drive in an uneven area, but to drive in an area adjacent to the uneven area (e.g., a flat area adjacent to the uneven area), as there is a risk of an accident occurring if the robot device (100) drives in an uneven area. Accordingly, at least one processor (133) may identify the driving level of the uneven area as avoidance driving based on map information, and may set the movement path of the robot device (100) so as not to drive in the uneven area.

[0173] According to an embodiment, the driving level information includes a driving level corresponding to each of a plurality of sub-spaces, and at least one processor (133) can control the driving of the robot device (100) based on the driving level of a sub-space corresponding to the location of the robot device (100) among the plurality of sub-spaces obtained from the driving level information.

[0174] FIG. 8 is a drawing for explaining an object loaded on a robot device according to an embodiment of the present disclosure.

[0175] According to an embodiment, an object can be loaded onto a robot device (100), and the robot device (100) can move the loaded object.

[0176] According to an embodiment, the robot device (100) includes a weight detection sensor, and at least one processor (133) can detect the weight of an object loaded on the robot device (100) through the weight detection sensor.

[0177] According to an embodiment, the robot device (100) can identify one of a plurality of driving levels based on the weight of the loaded object.

[0178] For example, at least one processor (133) may operate the robot device (100) in a first driving level among a plurality of driving levels when the weight of the loaded object is less than a threshold value or there is no loaded object.

[0179] For example, at least one processor (133) can control the robot device (100) to drive at a first driving level (e.g., high-speed driving, normal driving, etc.) so that there is no risk of an accident occurring due to a loaded object falling while the robot device (100) is driving if there is no object loaded on the robot device (100).

[0180] For example, if there is no object loaded on the robot device (100), at least one processor (133) can control the robot device (100) to drive on both the flat area and the uneven area at the first driving level, so that there is no risk of the object loaded on the robot device (100) falling even if the robot device (100) drives on an uneven area.

[0181] For example, at least one processor (133) can operate the robot device (100) in a second driving level among a plurality of driving levels if the weight of the loaded object is greater than a threshold value.

[0182] According to an embodiment, at least one processor (133) may control the robot device (100) to drive at a first driving level in a first sub-space corresponding to a flat area among a plurality of sub-spaces when the weight of an object loaded on the robot device (100) is greater than a threshold value, and may control the robot device (100) to drive at a second driving level in a second sub-space corresponding to an uneven area.

[0183] For example, at least one processor (133) may control the robot device (100) to drive on an uneven area at a second driving level (e.g., low-speed driving) because there is a risk that an object loaded on the robot device (100) may fall due to vibrations that occur while the robot device (100) drives on an uneven area.

[0184] However, it is not limited thereto, and at least one processor (133) may control the robot device (100) to drive at the second driving level in both the flat area and the uneven area, since there is a risk that the object loaded on the robot device (100) may fall even when the robot device (100) drives on a flat area if the weight of the object loaded on the robot device (100) is greater than a threshold value.

[0185] According to an embodiment, at least one processor (133) may set the uneven area to a fourth driving level (e.g., avoidance driving) so as not to drive in the uneven area if the weight of the loaded object is greater than a threshold value.

[0186] For example, if the weight of the loaded object is greater than a threshold value, at least one processor (1330) may set the uneven area to the fourth driving level (e.g., avoidance driving) so that the robot device (100) does not drive in the uneven area, and instead drives to an adjacent flat area, since there is a risk that the robot device (100) may malfunction or the object may fall if the robot device (100) drives in the uneven area.

[0187] For example, at least one processor (1330) may set the uneven area to a fourth driving level (e.g., avoidance driving) so as not to drive in the uneven area when the weight of the loaded object is greater than a threshold value and the height change of the uneven area exceeds a threshold range (e.g., when the height change exceeds 5 cm), and may drive in an adjacent flat area without driving in the uneven area.

[0188] FIG. 9 is a drawing for explaining evasive driving of a robot device according to one embodiment of the present disclosure.

[0189] According to an embodiment, at least one processor (133) can obtain a driving path (or movement path) for the robot device (100) to drive within a space based on map information.

[0190] For example, at least one processor (133) can identify each of the plurality of sub-spaces as a flat area or a non-flat area.

[0191] According to an embodiment, at least one processor (133) may obtain a movement path in which, if a first sub-space among a plurality of sub-spaces corresponds to a flat area, the first sub-space is driven at a first driving level among a plurality of driving levels, and if a second sub-space corresponds to an uneven area, the second sub-space is driven at a second driving level.

[0192] According to an embodiment, at least one processor (133) may adjust the movement path by using the height change of the floor identified based on a plurality of depth images received from the depth camera.

[0193] For example, at least one processor (133) can obtain a movement path for the robot device (100) to avoid an uneven area where the height change of the floor is greater than a threshold height.

[0194] For example, at least one processor (133) may set a movement path to avoid an uneven area where the height change is greater than or equal to a critical height, by using a drive unit provided in the robot device (100) (or, where climbing is impossible), or where an object loaded on the robot device (100) may fall. According to an embodiment, at least one processor (133) may set a movement path to avoid an uneven area where the height change is greater than or equal to a critical height.

[0195] According to an embodiment, at least one processor (133) may adjust the movement path of the robot device (100) using the pose of the robot device (100) identified based on the third sensing data received from the IMU.

[0196] For example, if at least one of the rotation angles of the robot device (100) relative to the x-axis (longitudinal axis), the rotation angles relative to the y-axis (horizontal axis), and the rotation angles relative to the z-axis (vertical axis) exceeds a critical angle, the robot device (100) may set a movement path to allow the robot device (100) to perform an avoidance drive, as there is a risk of an object loaded on the robot device (100) falling.

[0197] Returning to FIG. 2, the robotic device (100) according to an embodiment of the present disclosure may include a camera.

[0198] According to an embodiment, the camera photographs the floor corresponding to the position of the robot device (100), and at least one processor (133) can analyze the image received from the camera to obtain material information of the floor.

[0199] According to an embodiment, at least one processor (133) can control the movement of the robot device (100) based on material information.

[0200] For example, at least one processor (133) can capture a plurality of images by taking pictures of the floor at preset time intervals through a camera, and analyze each of the plurality of images to obtain material information corresponding to the floor of the space.

[0201] For example, floor materials can be categorized into those with a high coefficient of friction and those with a low coefficient of friction. Floors can be composed of a variety of materials, including carpet, tile, sheet, linoleum, wood, epoxy, urethane, marble, and PVC flooring. The coefficient of friction varies depending on the material used.

[0202] The floor material may be determined based on the material (or materials) used on the floor of each of the multiple sub-spaces. For example, the floor material of a sub-space may be implemented as a back.

[0203] According to an embodiment, at least one processor (133) analyzes an image of the floor captured by a camera to identify the material of the floor, and if the identified material is a slippery material (e.g., a material having a friction coefficient below a threshold value), the driving level for driving the robot device (100) can be adjusted.

[0204] For example, at least one processor (133) can control the robot device (100) to drive at a first driving level while driving in the first sub-space if the first sub-space among the plurality of sub-spaces corresponds to a flat area.

[0205] According to an embodiment, if the material of the floor constituting the first sub-space is a slippery material having a friction coefficient lower than a threshold value, there is a risk that the robot device (100) may deviate from the movement path (e.g., slipping of the robot device (100)) while moving in the first sub-space, or an object loaded on the robot device (100) may fall, and therefore, the robot device (100) may be controlled to drive at a second driving level while moving in the first sub-space, or the movement path may be adjusted to avoid driving in the first sub-space. For example, the movement speed of the robot device (100) according to the second driving level may be slower than the movement speed of the robot device (100) according to the first driving level.

[0206] According to an embodiment, at least one processor (133) can obtain information on the material of the floor by inputting an image of the floor captured by a camera into a neural network model.

[0207] For example, a neural network model may be a model trained to output information on the material of a floor when an image of the floor is input, using multiple sample images of floors of various materials as learning data.

[0208] The artificial intelligence-related function according to the present disclosure is operated through at least one processor (133) and memory of the robot device (100).

[0209] At this time, at least one processor may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors described above.

[0210] CPUs are general-purpose processors capable of performing not only general calculations but also artificial intelligence calculations. Their multi-layered cache structure allows for the efficient execution of complex programs. CPUs are advantageous for serial processing, enabling organic linking of previous and subsequent calculation results through sequential calculations. General-purpose processors are not limited to the examples described above, except where specifically identified as CPUs.

[0211] A GPU is a processor designed for large-scale computations, such as floating-point operations used in graphics processing. It integrates a large number of cores to perform large-scale computations in parallel. In particular, GPUs may be advantageous over CPUs in parallel processing methods, such as convolution operations. Furthermore, GPUs can be used as coprocessors to supplement the functions of CPUs. Processors for large-scale computations are not limited to the examples described above, except in cases where they are specifically referred to as GPUs.

[0212] An NPU is a processor specialized in artificial intelligence computation using artificial neural networks, and each layer of the artificial neural network can be implemented in hardware (e.g., silicon). Since an NPU is designed specifically according to the company's specifications, it has less freedom than a CPU or GPU, but can efficiently process the AI ​​computations requested by the company. Meanwhile, as a processor specialized in artificial intelligence computation, an NPU can be implemented in various forms, such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), or a Vision Processing Unit (VPU). Except as specifically stated as an NPU, an AI processor is not limited to the examples described above.

[0213] Additionally, one or more processors may be implemented as a System on Chip (SoC). In this case, in addition to one or more processors, the SoC may further include memory and a network interface, such as a bus, for data communication between the processor and the memory.

[0214] When a plurality of processors are included in a SoC (System on Chip) included in a robot device (100), the robot device (100) can perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the plurality of processors. For example, the robot device (100) can perform operations related to artificial intelligence by using at least one of a GPU, NPU, VPU, TPU, or hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors. However, this is merely an example, and it is of course possible to process operations related to artificial intelligence by using a CPU or other general-purpose processor.

[0215] In addition, the robot device (100) can perform operations related to functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in at least one processor (133). In particular, the robot device (100) can perform artificial intelligence operations such as convolution operations, matrix multiplication operations, etc. in parallel by utilizing multiple cores included in the processor (133).

[0216] One or more processors (133) are controlled to process input data according to predefined operation rules or artificial intelligence models stored in memory. The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0217] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0218] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0219] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0220] FIG. 10 is a flowchart for explaining a method for controlling a robot device according to an embodiment of the present disclosure.

[0221] Referring to FIG. 10, a method for controlling a robot device for obtaining map information obtains floor information of a space where the robot device is located based on first sensing data received from a first lidar sensor and second sensing data received from a second lidar sensor (S1010).

[0222] Multiple sub-spaces included in a space are identified based on floor information (S1020).

[0223] Driving level information including driving levels corresponding to each of multiple sub-spaces is acquired (S1030).

[0224] The floor information according to the embodiment includes information related to flat areas and uneven areas within a space, and the operation S1020 for identifying a plurality of sub-spaces can identify a plurality of sub-spaces by dividing the space into flat areas or uneven areas based on the floor information.

[0225] The operation S1030 for obtaining driving level information includes an operation of identifying the driving level of the first sub-space as the first driving level when the floor of the first sub-space among the plurality of sub-spaces corresponds to a flat area, and an operation of identifying the driving level of the second sub-space as the second driving level when the floor of the second sub-space among the plurality of sub-spaces corresponds to an uneven area, and the driving speed of the robot device according to the second level may be relatively slower than the driving speed of the robot device according to the first level.

[0226] According to an embodiment, the operation S1020 for identifying a plurality of sub-spaces may include an operation for identifying an area where the robot device is located as an uneven area when the distance between the first lidar sensor included in the first sensing data and the floor where the robot device is located is repeatedly increased and decreased, and the distance between the second lidar sensor included in the second sensing data and the floor is repeatedly increased and decreased.

[0227] According to an embodiment, an operation of identifying an area where a robot device is located as an uneven area may include an operation of identifying an area where a robot device is located as an uneven area in the first time interval, based on first sensing data and second sensing data, if an increase / decrease pattern or a change in the distance from the floor sensed in a first direction in a first time interval does not correspond to a moving speed of the robot device, or if an increase / decrease pattern or a change in the distance from the floor sensed in a second direction does not correspond to a moving speed of the robot device.

[0228] A control method according to an embodiment of the present disclosure may include an operation of controlling a depth camera to capture a floor corresponding to a position of a robot device at preset time intervals to obtain a plurality of depth images, and an operation of identifying a plurality of sub-spaces included in a space based on the plurality of depth images.

[0229] A control method according to an embodiment of the present disclosure further includes an operation of detecting a pose of a robot device based on third sensing data received from an IMU (Inertial Measurement Unit), and the operation S1020 of identifying a plurality of sub-spaces includes an operation of identifying a plurality of sub-spaces based on the pose and floor information of the robot device, and the third sensing data may include at least one of a roll, a pitch, or a yaw indicating the pose of the robot device.

[0230] The S1020 operation for identifying multiple sub-spaces may include an operation for identifying a pose of a robot device as tilted if at least one of roll, pitch, or yaw is greater than or equal to a threshold angle, and an operation for identifying an area in which a robot device identified as tilted is located as an uneven area.

[0231] A control method according to an embodiment of the present disclosure may include an operation of controlling a camera to capture a floor corresponding to a position of a robot device to obtain an image, an operation of obtaining material information of the floor based on the image, and an operation of obtaining driving level information including a driving level corresponding to each of a plurality of sub-spaces based on the material information.

[0232] FIG. 11 is a flowchart illustrating a method for controlling the driving of a robot device according to an embodiment of the present disclosure.

[0233] Referring to FIG. 11, a control method of a robot device for controlling the driving of a robot device obtains a driving level of a sub-space corresponding to a position of the robot device among a plurality of sub-spaces from driving level information (S1110).

[0234] The driving of the robot device is controlled based on the acquired driving level (S1110).

[0235] A control method according to an embodiment of the present disclosure may further include an operation of controlling a depth camera to capture a floor corresponding to a position of a robot device at preset time intervals to obtain a plurality of depth images, an operation of identifying a change in the height of a floor of a sub-space corresponding to an uneven area among a plurality of sub-spaces based on the plurality of depth images, and an operation of controlling the driving of the robot device based on the identified change in height.

[0236] A control method according to an embodiment of the present disclosure may further include an operation of detecting a pose of a robot device based on third sensing data received from an IMU (Inertial Measurement Unit) and an operation of controlling driving of the robot device based on the pose of the robot device.

[0237] A control method according to an embodiment of the present disclosure may further include an operation of controlling a camera to capture a floor corresponding to a position of a robot device to obtain an image, an operation of obtaining material information of the floor based on the image, and an operation of controlling the driving of the robot device based on the material information.

[0238] A control method according to an embodiment of the present disclosure may include an operation of detecting a weight of an object loaded on a robot device, an operation of controlling driving of the robot device by identifying a driving level corresponding to a sub-space corresponding to a position of the robot device among a plurality of sub-spaces when the detected weight is greater than or equal to a threshold value, and an operation of controlling driving of the robot device without changing the driving level when the detected weight is less than the threshold value.

[0239] It goes without saying that the various embodiments of the present disclosure can be applied not only to robotic devices but also to various types of electronic devices.

[0240] Meanwhile, the various embodiments described above may be implemented in a computer-readable recording medium or similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0241] Meanwhile, computer instructions for performing processing operations of a robot device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When the computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, the computer instructions cause the specific device to perform processing operations in the robot device (100) according to various embodiments described above.

[0242] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0243] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In a robot device, memory; A first LiDAR sensor configured to detect obstacles in a first direction; A second lidar sensor configured to detect obstacles in a second direction opposite to the first direction; and Based on the first sensing data received from the first lidar sensor and the second sensing data received from the second lidar sensor, floor information about the floor of the space where the robot device is located is acquired, Identifying multiple sub-spaces included in the space based on the above floor information, Obtaining driving level information including driving levels related to each of the plurality of sub-spaces, and storing the driving level information in the memory, A robot device that controls the movement of the robot device based on the driving level of a sub-space corresponding to the location of the robot device among the plurality of sub-spaces, obtained from the driving level information stored in the memory.

2. In paragraph 1, The above floor information is, Contains information related to flat and uneven areas within the above space, At least one processor of the above, A robot device that identifies the plurality of sub-spaces by dividing the space into the flat area or the uneven area based on the floor information.

3. In paragraph 2, At least one processor of the above, If the floor of the first sub-space among the above plurality of sub-spaces corresponds to the flat area, the driving level of the first sub-space is identified as the first driving level. If the floor of the second sub-space among the above multiple sub-spaces corresponds to the non-flat area, the driving level of the second sub-space is identified as the second driving level. In the above second driving level, the driving speed of the robot device is: A robotic device having a driving speed slower than that of the robotic device at the first driving level.

4. In paragraph 1, At least one processor of the above, Based on the variation of the first distance and the second distance, the area where the robot device is located is identified as an uneven area, The above first distance is, The distance between the first lidar sensor and the floor where the robot device is located, The above second distance is, A robotic device, the distance between the second lidar sensor and the floor.

5. In paragraph 4, At least one processor of the above, A robot device that identifies the area where the robot device is positioned in the first time interval as the uneven area, if, based on the first sensing data and the second sensing data, a first pattern of increase / decrease in the first distance from the floor detected in the first direction in the first time interval does not correspond to a driving speed of the robot device, or a second pattern of increase / decrease in the second distance from the floor detected in the second direction does not correspond to a driving speed of the robot device.

6. In paragraph 1, Including a depth camera; At least one processor of the above, Using the depth camera, multiple depth images are acquired to capture the floor corresponding to the position of the robot device at preset time intervals, Based on the plurality of depth images, the change in the height of the floor of a sub-space corresponding to an uneven area among the plurality of sub-spaces is identified, A robotic device that controls movement of the robotic device based on the change in height.

7. In paragraph 1, It further includes an IMU (Inertial Measurement Unit); At least one processor of the above, Detecting the pose of the robot device based on the third sensing data received from the IMU, Identifying the plurality of sub-spaces based on the detected pose and the floor information, The above third sensing data is, A robotic device comprising at least one of roll, pitch or yaw representing a pose of the robotic device.

8. In paragraph 7, At least one processor of the above, If at least one of the above roll, pitch or yaw is greater than a critical angle, the pose of the robot device is identified as tilted. A robotic device that identifies an area identified by the above inclination as an uneven area.

9. In paragraph 1, Including cameras; At least one processor of the above, Using the camera to acquire an image so as to photograph the floor corresponding to the position of the robot device, Obtain material information of the floor based on the image above, A robotic device that controls the movement of the robotic device based on the material information.

10. In paragraph 1, At least one processor of the above, Detects the weight of an object loaded on the above robot device, If the detected weight is greater than a threshold value, the driving level of the sub-space corresponding to the position of the robot device is identified to control the movement of the robot device, A robotic device that controls movement of the robotic device without changing the driving level if the detected weight is less than the threshold value.

11. In a method for controlling a robot device, A step of receiving first sensing data from a first lidar sensor in a first direction based on the above robot device; A step of receiving second sensing data from a second lidar sensor in a second direction opposite to the first direction; A step of obtaining floor information about the floor of a space where the robot device is located based on the first sensing data and the second sensing data; A step of identifying a plurality of sub-spaces included in the space based on the floor information; A step of obtaining driving level information including a driving level related to each of the plurality of sub-spaces; and A control method further comprising: a step of controlling the driving of the robot device based on a driving level of a sub-space corresponding to a position of the robot device among the plurality of sub-spaces.

12. In paragraph 11, The above floor information is, Contains information related to flat and uneven areas within the above space, The step of identifying the above multiple sub-spaces is A control method, comprising: a step of identifying the plurality of sub-spaces by dividing the space into the flat area or the uneven area based on the floor information.

13. In paragraph 12, The step of obtaining the above driving level information is: If the floor of the first sub-space among the plurality of sub-spaces corresponds to the flat area, a step of identifying the driving level of the first sub-space as the first driving level; and A step of identifying the driving level of the second sub-space as the second driving level when the floor of the second sub-space among the plurality of sub-spaces corresponds to the non-flat area; In the above second driving level, the driving speed of the robot device is: A control method, wherein the driving speed of the robot device is slower than that of the robot device at the first driving level.

14. In paragraph 11, The step of identifying the above multiple sub-spaces is: A step of identifying an area where the robot device is located as an uneven area based on the variation of the first distance and the second distance; The above first distance is, The distance between the first lidar sensor and the floor where the robot device is located, The above second distance is, A control method, wherein the distance between the second lidar sensor and the floor is:

15. In paragraph 14, The step of identifying the area where the above robot device is located as the above uneven area is, A control method comprising: a step of identifying the area where the robot device is positioned in the first time interval as the uneven area, if, based on the first sensing data and the second sensing data, a first pattern of increase / decrease in the first distance from the floor detected in the first direction in the first time interval does not correspond to a driving speed of the robot device, or a second pattern of increase / decrease in the second distance from the floor detected in the second direction does not correspond to a driving speed of the robot device.

Images