Robot apparatus for identifying non-flat area and

Abstract

A robot apparatus is disclosed. The robot device includes: a memory; a first LiDAR sensor configured to detect an obstacle in a first direction; and a second LiDAR sensor configured to detect an obstacle in a second direction opposite to the first direction, in which the robot device is configured to obtain ground information on a ground of a space in which the robot device is located based on first sensing data received from the first LiDAR sensor and second sensing data received from the second LiDAR sensor, identifying a plurality of subspaces included in a space based on ground information, obtaining driving level information including a driving level associated with each of the plurality of subspaces, storing the driving level information in a memory, and controlling movement of the robot device based on a travel level of a subspace corresponding to a position of the robot device among the plurality of subspaces obtained from the travel level information stored in the memory.

Description

TECHNICAL FIELD

[0001] The disclosure relates to a robot device and a control method thereof, and more particularly, to a robot device that recognizes a flat area and a non-flat area in a space and a control method thereof. BACKGROUND

[0002] Recently, the distribution of robot devices is accelerating in various fields, and robot devices are not only replacing the role of people, but recently, the trend is that unmanned factories (or smart factories) and the like that are operated only with robot devices without people are rapidly increasing.

[0003] When a robot device autonomously travels in a factory, it is necessary to detect the surrounding environment of the robot device with high precision so that the robot device can move more easily.

[0004] For example, it is necessary to detect whether the floor on which the robot device travels is flat (for example, whether the floor is bumpy), and when the robot device travels on an uneven floor, it is necessary for the robot device to travel safely to prevent a drop accident of an object loaded on the robot device from occurring.

[0005] In the past, a robot device merely detected an obstacle and traveled to avoid the obstacle, and there was a problem in that it was difficult to adjust the travel speed of the robot device or make the robot device travel to avoid the obstacle in consideration of the state of the floor (for example, whether there is a bumpy curve, etc.).

[0006] Therefore, there are various needs for a method in which a robot device located in a factory or the like detects the state of the floor with higher precision and adjusts the travel speed or travel route in consideration of the state of the floor. SUMMARY

[0007] TECHNICAL SOLUTION A robot device according to an embodiment of the disclosure includes a memory, a first LiDAR sensor configured to detect an obstacle in a first direction, a second LiDAR sensor configured to detect an obstacle in a second direction opposite to the first direction, and at least one processor. The at least one processor is configured to: obtain floor information about a floor of a space in which the robot device is located, based on first sensing data received from the first LiDAR sensor and second sensing data received from the second LiDAR sensor; identify a plurality of sub-spaces included in the space based on the floor information; obtain travel level information and store the travel level information in the memory, wherein the travel level information includes a travel level associated with each of the plurality of sub-spaces; and control movement of the robot device based on a travel level of a sub-space corresponding to a location of the robot device, wherein the sub-space is among the plurality of sub-spaces, and wherein the travel level is obtained from the travel level information stored in the memory.

[0008] A control method of a robot device according to an embodiment of the disclosure includes: receiving first sensing data in a first direction based on a robot device from a first LiDAR sensor; receiving second sensing data in a second direction opposite to the first direction from a second LiDAR sensor; obtaining floor information about a floor of a space in which the robot device is located, based on the first sensing data and the second sensing data; identifying a plurality of sub-spaces included in the space based on the floor information; obtaining travel level information, wherein the travel level information includes a travel level associated with each of the plurality of sub-spaces; and controlling travel of the robot device based on a travel level of a sub-space corresponding to a location of the robot device, wherein the sub-space is among the plurality of sub-spaces.

[0009] In a non-transitory computer-readable recording medium including a program that performs a control method of a robot device according to an embodiment of the disclosure, the control method of the robot device includes: receiving first sensing data in a first direction based on a robot device from a first LiDAR sensor; receiving second sensing data in a second direction opposite to the first direction from a second LiDAR sensor; obtaining floor information about a floor of a space in which the robot device is located, based on the first sensing data and the second sensing data; identifying a plurality of sub-spaces included in the space based on the floor information; obtaining travel level information, wherein the travel level information includes a travel level associated with each of the plurality of sub-spaces; and controlling travel of the robot device based on a travel level of a sub-space corresponding to a location of the robot device, wherein the sub-space is among the plurality of sub-spaces.

[0010] According to an embodiment of the disclosure, in a device for controlling a robot device, the device includes at least one memory configured to store program code, and at least one processor configured to read the program code and operate as instructed by the program code. The program code includes first receiving code configured to cause the at least one processor to receive first sensing data in a first direction based on a robot device from a first LiDAR sensor, second receiving code configured to cause the at least one processor to receive second sensing data in a second direction opposite to the first direction from a second LiDAR sensor, first obtaining code configured to cause the at least one processor to obtain ground information about a ground of a space in which the robot device is located based on the first sensing data and the second sensing data, first identifying code configured to cause the at least one processor to identify a plurality of sub-spaces included in the space based on the ground information, second obtaining code configured to cause the at least one processor to obtain travel level information, wherein the travel level information includes a travel level associated with each of the plurality of sub-spaces, and first control code configured to cause the at least one processor to control travel of the robot device based on the travel level of a sub-space corresponding to a position of the robot device, among the plurality of sub-spaces. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a diagram for illustrating travel of a robot device according to an embodiment of the disclosure; Figure 2 is a block diagram of a robot device according to an embodiment of the disclosure; Figure 3 is a diagram for illustrating a view associated with a LiDAR sensor according to an embodiment of the disclosure; Figure 4 is a diagram for illustrating data from one or more LiDAR sensors according to an embodiment of the disclosure; Figure 5 is a diagram illustrating a depth image according to an embodiment of the disclosure; Figure 6a is a diagram illustrating a change in pitch of a robot device according to an embodiment of the disclosure; Figure 6b is a diagram illustrating a change in roll of a robot device according to an embodiment of the disclosure; Figure 7 is a diagram illustrating travel and / or movement speed of a robot device according to an embodiment of the disclosure; Figure 8 is a diagram illustrating an object loaded on a robot device according to an embodiment of the disclosure; Figure 9is a diagram illustrating evasive travel of a robot device according to an embodiment of the disclosure; Figure 10 is a flowchart illustrating a method of controlling a robot device according to an embodiment of the disclosure; and Figure 11 is a flowchart illustrating a method of controlling a robot device according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0012] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0013] As terms used in embodiments of the present disclosure, the general terms which are currently widely used in consideration of functions in the present disclosure are selected as far as possible. However, the terms can vary according to the intention of those skilled in the art, the court's previous judgment, or the appearance of new technology at the time of application, etc. Also, in a certain case, there can be a term specially designated by the applicant, and in this case, the meaning of the term will be described in the relevant description of the present disclosure in detail. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the overall content of the present disclosure, not based on only the names of the terms.

[0014] Also, in the present specification, expressions such as "have", "may have", "include", and "may include" indicate the presence of such features (for example: elements such as numbers, functions, operations, and components), and do not exclude the presence of additional features.

[0015] Also, the expression "at least one of A and / or B" should be interpreted to mean any one of "A" or "B" or "A and B".

[0016] Also, the expressions "first", "second", and the like used in the present specification can be used to describe various elements regardless of the order and / or importance of the elements. Also, such expressions are used only to distinguish one element from another element, and are not intended to limit the elements.

[0017] Meanwhile, the description that one element (for example: a first element) in the present disclosure is "coupled with / to" or "connected to" another element (for example: a second element) should be interpreted as including the case in which the one element is directly coupled to the other element and the case in which the one element is coupled to the other element through yet another element (for example: a third element).

[0018] Further, the singular expression includes the plural expression, unless it is clear from the context that it is differently defined. Further, in the disclosure, terms such as "include" or "consist of" should be interpreted as specifying the presence of such features, numbers, steps, operations, elements, components, or combinations thereof described in the specification, but should not be interpreted as precluding the presence or possibility of adding one or more other features, numbers, steps, operations, elements, components, or combinations thereof.

[0019] In addition, in the disclosure, a "module" or a "part" performs at least one function or operation and can be implemented as hardware or software, or as a combination of hardware and software. Further, a plurality of "modules" or "parts" can be integrated as at least one module and implemented as at least one processor (not shown) except for a "module" or a "part" that needs to be implemented as a specific hardware.

[0020] Further, in the present specification, the term "user" can refer to a person using an electronic device or a device (for example: an artificial intelligence electronic device) using an electronic device.

[0021] Hereinafter, embodiments of the disclosure will be described in greater detail with reference to the accompanying drawings.

[0022] Figure 1 is a view for illustrating travel of a robot device according to an embodiment of the disclosure.

[0023] According to Figure 1 As illustrated, the robot device 100 can represent various forms of devices having the ability to perform functions by itself. As an example, the robot device 100 can include a smart device that senses a surrounding environment of the robot device 100 in real time based on sensing data of a sensor (for example, a Light Detection and Ranging (LiDAR) sensor) and a camera (for example, a depth camera, an RGB camera, or the like), and collects information and operates autonomously in addition to performing a simple repetitive function.

[0024] The robot device 100 according to an embodiment of the disclosure can include a driver including an actuator or a motor. The driver according to an embodiment can include a wheel, a brake, or the like, and the robot device 100 can move in space by itself by using the wheel, the brake, or the like included in the driver.

[0025] The robot device 100 according to an embodiment can include a robot joint (a link or a joint). In the disclosure, the robot joint can mean a component of the robot device 100 for replacing the function of a human's arm or hand.

[0026] The robot device 100 according to an embodiment of the disclosure can include a sensor, and obtain map information corresponding to a space (e.g., a predetermined space inside a factory, a predetermined space in a home) in which the robot device 100 is located, based on sensing data of the sensor.

[0027] For example, the robot device 100 can include a LiDAR sensor, and generate a 2D or 3D map corresponding to a space by using sensing data of the LiDAR sensor and various mapping algorithms.

[0028] A light detection and ranging (LiDAR) sensor is a sensor that measures the distance of a certain object. Specifically, the LiDAR sensor can perform sensing by a method of emitting light by using a light source and detecting reflected light by using a sensor around the light source when the light is emitted (or radiated) by a target object. The robot device 100 can measure the time taken until the light returns, and calculate the distance to the target object based on the measured time and the speed of light. The robot device 100 can generate a 3D (or 2D) map of a certain space by repeatedly measuring in the certain space using the LiDAR sensor. Here, the map can include an image or information in a 2D or 3D form indicating the location, size, shape, etc. of the entire space and a plurality of sub-spaces included in the space. The map can also include an image or information in a 2D or 3D form indicating the location, size, shape, etc. of obstacles (e.g., factory equipment (e.g., a production machine), furniture, home appliances, etc.) included in each of the plurality of sub-spaces. The map information can be various information or data used to constitute such a map.

[0029] The robot device 100 according to an embodiment can identify the location of the robot device 100 in the map based on the sensing data.

[0030] As an example, the robot device 100 can identify a movement route based on the map information, and identify the location of the robot device 100 in the map while the robot device 100 is moving in the space according to the movement route.

[0031] As an example, the robot device 100 can obtain map information corresponding to a space in which the robot device 100 is located by performing a simultaneous localization and mapping (SLAM) operation, and identify the current location of the robot device 100 in the map.

[0032] In addition, the robot device 100 can be classified into industrial use, medical use, home use, military use, and exploration use, according to functions that the robot device 100 can perform. According to an embodiment, the robot device for industrial use can be subdivided into a robot device used in a product manufacturing process in a factory, a robot device that performs customer service, receives an order and service, etc. in a shop or a restaurant, etc. For example, the robot device 100 can be implemented as a service robot device that can deliver a service item to a location desired by a user, or a specific location of various places such as a restaurant, a hotel, a shopping mall, a hospital, a clothing store, etc.

[0033] However, the present disclosure is not limited thereto, and the robot device 100 can be classified in various ways according to application fields, functions, and use purposes. For example, the robot device 100 can be implemented in various forms such as a movable projector equipped with a projection function, a robot cleaner including a cleaning function, a service robot for delivering or transporting various goods, etc. According to an embodiment, the robot device 100 can further include various components such as a projection part (not shown), a dry cleaning module, a wet cleaning module, a robot arm, etc., but detailed illustrations and explanations regarding this aspect will be omitted.

[0034] According to an embodiment of the present disclosure, the robot device 100 can obtain information about a floor of a space based on sensing data. In addition, according to an embodiment, the robot device 100 can divide the space into a plurality of sub-spaces based on the floor information.

[0035] For example, the floor information can include information about a flat area in the space and information about a non-flat area in the space.

[0036] As an example, the flat area can be an area in which uniformity of a floor surface is within a certain allowable range. The flat area can also be referred to as a low-risk area, an area in which movement of the robot device 100 is easy, an area in which a risk is less, etc., but will be generally referred to as a flat area hereinafter.

[0037] As an example, the non-flat area can be an area in which uniformity of a floor surface exceeds an allowable range, i.e., an area in which a floor is not flat beyond a tolerable range. The non-flat area can also be referred to as a high-risk area, an area in which movement of the robot device 100 is not easy, a dangerous area, or a bumpy area, etc., but will be generally referred to as a non-flat area hereinafter.

[0038] As an example, the information about the flat area can include a position, a size (or an area), a shape, etc. of the flat area inside the space, and the information about the non-flat area can include a position, a size, a shape, etc. of the non-flat area inside the space.

[0039] According to an embodiment, the robot device 100 can obtain map information that divides a space into a plurality of sub-spaces based on floor information.

[0040] According to an embodiment, the robot device 100 can control travel of the robot device 100 by identifying whether a sub-space in which the robot device 100 is to move among the plurality of sub-spaces is a flat area or a non-flat area based on the map information.

[0041] According to an embodiment, the robot device 100 can divide a space into a plurality of sub-spaces by identifying flat areas and non-flat areas in the space, or divide the space into a plurality of sub-spaces by identifying independent areas surrounded by walls. However, this is merely an example, and the robot device 100 can divide the space into a plurality of sub-spaces by identifying furniture, steps, door thresholds, etc. in the space.

[0042] Figure 2 is a block diagram for illustrating a robot device according to an embodiment of the disclosure.

[0043] Referring to Figure 2 , the robot device 100 includes a sensor 110, a driver 120, and a main module 130.

[0044] According to an embodiment, the sensor 110 can include a plurality of LiDAR sensors, such as a first LiDAR sensor 111, a second LiDAR sensor 112. The sensor 110 can further include a camera 113. The main module 130 can include a communication interface 131, a memory 132 (including one or more memory chips and types), at least one processor 133, and a controller 134.

[0045] The sensor 110 is a component for sensing various information. According to an embodiment, the at least one processor 133 can obtain various information based on a sensing value of the sensor 110. For example, information obtained by the sensor 110 can include an image and depth information. The image can include an RGB value of each of a plurality of pixels included in the image. The depth information can include a depth map including a depth value of each of the plurality of pixels.

[0046] According to an embodiment, the sensor 110 includes the first LiDAR sensor 111 and the second LiDAR sensor 112.

[0047] Each of the first LiDAR sensor 111 and the second LiDAR sensor 112 can emit light toward a target object by using a light source, and detect light reflected from the target object. The at least one processor 133 can identify a distance to the target object based on a time taken until the light is reflected from the target object and detected after the light is emitted from each of the first LiDAR sensor 111 and the second LiDAR sensor 112.

[0048] According to an embodiment of the disclosure, the first LiDAR sensor 111 can be disposed to detect a first direction based on the robot device 100 (or, emit light in the first direction). In the present embodiment, the first direction can be a traveling direction of the robot device 100 when the robot device 100 travels forward, or a direction toward a front side of the robot device 100.

[0049] For example, the first LiDAR sensor 111 can detect a distance to an obstacle located at the front side of the robot device 100 when the robot device 100 travels forward. However, this is merely an example, and the first direction can correspond to a side (e.g., a right side or a left side, etc.) of the robot device 100.

[0050] According to an embodiment of the disclosure, the second LiDAR sensor 112 can be disposed to detect a second direction based on the robot device 100 opposite to the first direction (or, emit light in the second direction). In an embodiment, the second direction can be a direction opposite to a traveling direction of the robot device 100 when the robot device 100 travels forward, or a direction corresponding to a rear side of the robot device 100.

[0051] For example, the second LiDAR sensor 112 can detect a distance to an obstacle located at the rear side of the robot device 100 when the robot device 100 travels forward. However, this is merely an example, and the second direction can correspond to a side (e.g., a right side or a left side, etc.) of the robot device 100. Also, the first direction and the second direction can be implemented as directions completely opposite to each other, but are not necessarily limited thereto, and they can be implemented as two different directions related at a certain angle or more.

[0052] According to an embodiment, the first LiDAR sensor 111 can obtain first sensing data that senses a distance to an obstacle located in the first direction, and the second LiDAR sensor 112 can obtain second sensing data that senses a distance to an obstacle located in the second direction.

[0053] According to an embodiment, the sensor 110 can include a camera 113. The camera 113 can include a stereo camera, an RGB-D camera, a time-of-flight (ToF) camera, a depth camera, etc. However, the disclosure is not limited to this example, and the sensor 110 can include various sensors that can obtain an image and depth information.

[0054] The driver 120 can control the movement of the robot 200. For example, the driver 120 can control the movement of the robot device 100, stop the robot device 100 that is moving, and / or control the movement speed and / or movement direction of the robot device 100.

[0055] In an embodiment, the robot device 100 can move using any suitable driving device (e.g., a wheeled driving device, a walking driving device, etc.).

[0056] Wheeled refers to a method in which the robot device 100 moves by rotation of a wheel. If the robot device 100 is a wheeled robot, the robot device 100 can include one or more wheels. The driver 120 can include a device that generates a force for rotating the wheel. For example, the driver 120 can be implemented as a gasoline engine, a diesel engine, a liquefied petroleum gas (LPG) engine, or an electric motor, etc., depending on the fuel (or energy source) used.

[0057] Walking refers to a method in which the robot device 100 moves by movement of a leg. If the robot device 100 is walking (e.g., a biped walking robot, a triped walking robot, a quadruped walking robot, etc.), the robot device 100 can include two or more legs that support the robot device 100. The leg can include a plurality of links and a joint connected to the links. The driver 120 can include a device that generates a force that raises or lowers the leg by rotating the link about the joint as a center. For example, the driver 120 can be implemented as a motor and / or an actuator, etc.

[0058] In addition, the driver 120 can control the movement of a part of the robot device 100. The driver 120 can be coupled between a first part (e.g., a body) and a second part (e.g., a head, an arm, etc.) of the robot device 100. The driver 120 can rotate the second part. For example, the driver 120 can be implemented as a motor and / or an actuator, etc.

[0059] The main module 130 can be implemented as hardware and include a communication interface 131, a memory 132, at least one processor 133, and a controller 134.

[0060] The communication interface 131 can perform data communication with an electronic device through control of the at least one processor 133. For example, the communication interface 131 can include a communication circuit that can perform data communication between the robot device 100 and the electronic device by using at least one of data communication methods including a wired LAN, a wireless LAN, Wi-Fi, Bluetooth, Zigbee, Wi-Fi Direct (WFD), an Infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), Worldwide Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliance (WiGig), and RF communication.

[0061] In the memory 132, instructions, data structures, and program codes that can be read by the at least one processor 133 can be stored. Operations performed by the at least one processor 133 can be implemented by executing the instructions or program codes stored in the memory 132.

[0062] The memory 132 can include a flash memory type memory, a hard disk type memory, a micro multi-media card type memory, and a card type memory (e.g., an SD or XD memory, etc.), and can include a non-volatile memory and a volatile memory (such as a random access memory (RAM) or a static random access memory (SRAM)), in which the non-volatile memory includes at least one of a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, or an optical disk.

[0063] According to an embodiment, the at least one processor 133 controls the overall operation of the robot device 100. Specifically, the at least one processor 133 can be connected with various components of the robot device 100 and control the overall operation of the robot device 100.

[0064] The at least one processor 133 can perform operations of the robot device 100 according to various embodiments by executing at least one instruction stored in the memory 132.

[0065] The at least one processor 133 can include one or more of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a many integrated core (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. The at least one processor 133 can control one or any combination of other components of the robot device and perform operations regarding communication or data processing. The at least one processor 133 can execute one or more programs or instructions stored in the memory 132. For example, the at least one processor 133 can perform a method according to an embodiment of the disclosure by executing one or more instructions stored in the memory 132.

[0066] When a method according to an embodiment of the disclosure includes a plurality of operations, the plurality of operations can be performed by one processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, all of the first operation, the second operation, and the third operation can be performed by a first processor, or the first operation and the second operation can be performed by a first processor (e.g., a general-purpose processor), and the third operation can be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0067] The at least one processor 133 can be implemented as a single core processor including one core, or can be implemented as one or more multi-core processors including a plurality of cores (e.g., a plurality of cores of the same kind or a plurality of cores of different kinds). When the at least one processor 133 is implemented as a multi-core processor, each of the plurality of cores included in the multi-core processor can include an internal memory (such as a cache, an on-chip memory, etc.) of the processor, and a common cache shared by the plurality of cores can be included in the multi-core processor. Furthermore, each of the plurality of cores (or some of the plurality of cores) included in the multi-core processor can independently read program instructions for implementing a method according to an embodiment of the disclosure and execute the instructions, or all (or some of the cores) of the plurality of cores can be linked to each other and read program instructions for implementing a method according to an embodiment of the disclosure and execute the instructions.

[0068] When a method according to an embodiment of the disclosure includes a plurality of operations, the plurality of operations can be performed by one processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, all of the first operation, the second operation, and the third operation can be performed by a first processor, or the first operation and the second operation can be performed by a first processor (e.g., a general-purpose processor), and the third operation can be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0069] In embodiments of the disclosure, the processor can mean a system on chip (SoC) in which at least one processor and other electronic components are integrated, a single core processor, a multi-core processor, or a core included in a single core processor or a multi-core processor. Also, here, the core can be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, etc., but embodiments of the disclosure are not limited thereto.

[0070] According to embodiments of the disclosure, the at least one processor 133 can obtain information of a floor of a space in which the robot device 100 is located, based on first sensing data received from the first LiDAR sensor 111 and second sensing data received from the second LiDAR sensor 112.

[0071] The at least one processor 133 according to embodiments can identify a plurality of sub-spaces included in the space, based on the floor information.

[0072] The controller 134 can control components of the robot device 100. The controller 134 can control components (e.g., the sensors 110 and the driver 120, etc.) of the robot device 100 based on a signal provided from the at least one processor 133. For example, the controller 134 can generate a control signal by using the signal provided from the at least one processor 133, and provide the control signal to the components of the robot device 100. Accordingly, the components of the robot device 100 can perform operations corresponding to the operation results of the at least one processor 133. The controller 134 can be implemented as at least one IC (e.g., a controller IC).

[0073] Figure 3 is a diagram illustrating a LiDAR sensor according to an embodiment of the disclosure.

[0074] According to at least one embodiment, when the robot device 100 travels according to the control of the at least one processor 133, the at least one processor 133 can obtain map information of a space in which the robot device 100 is located according to any appropriate method (e.g., a SLAM operation), and identify a current position of the robot device 100 in the space.

[0075] Referring to Figure 3 In the scenario 300 of FIG. 11, if the robot device 100 travels in a flat area, light emitted by the first LiDAR sensor 111 is reflected on an obstacle (e.g., a wall, etc.) located in a first direction based on the robot device 100, and the light reflected on the obstacle can be received by the first LiDAR sensor 111.

[0076] According to an embodiment, the at least one processor 133 can identify a first distance to an obstacle located at a front side of the robot device 100 based on first sensing data received from the first LiDAR sensor 111.

[0077] Light emitted by the second LiDAR sensor 112 is reflected on an obstacle located in a second direction based on the robot device 100, and the light reflected on the obstacle can be received by the second LiDAR sensor 112.

[0078] According to an embodiment, the at least one processor 133 can identify a second distance to an obstacle located at a rear side of the robot device 100 based on second sensing data received from the second LiDAR sensor 112.

[0079] According to an embodiment, if the robot device 100 travels in a flat area, the first distance to the obstacle detected by the first LiDAR sensor 111 can regularly decrease, and the second distance to the obstacle detected by the second LiDAR sensor 112 can regularly increase.

[0080] According to an embodiment, if an obstacle is located at a front side of the robot device 100, and the robot device 100 moves in the first direction, the distance between the robot device 100 and the obstacle decreases to correspond to the moving speed of the robot device 100, and thus, the first distance to the obstacle detected by the first LiDAR sensor 111 can regularly decrease.

[0081] Further, if an obstacle is located at a rear side of the robot device 100, and the robot device 100 moves in the first direction, the distance between the robot device 100 and the obstacle thus increases to correspond to the moving speed of the robot device 100, and thus, the second distance to the obstacle detected by the second LiDAR sensor 112 can regularly increase.

[0082] According to an embodiment, when the robot device 100 travels, if the first distance to the obstacle located at the front side regularly decreases (e.g., to correspond to the moving speed of the robot device 100) based on the first sensing data received from the first LiDAR sensor 111, or the second distance to the obstacle located at the rear side regularly increases (e.g., to correspond to the moving speed of the robot device 100) based on the second sensing data received from the second LiDAR sensor 112, the at least one processor 133 can identify an area in which the robot device 100 moves according to the travel of the robot device 100 as a flat area.

[0083] The at least one processor 133 can obtain information related to a flat area including at least one of a location, a size, or a shape of an area identified as a flat area, in a case where the map information is obtained.

[0084] Referring to Figure 3 Scenarios 310 and 320, if at least one of a first distance from an obstacle detected by the first LiDAR sensor 111 or a second distance from an obstacle detected by the second LiDAR sensor 112 rapidly increases and decreases, the at least one processor 133 can identify an area in which the robot device 100 moves as a non-flat area.

[0085] As shown in Figure 3 When the robot device 100 travels in a non-flat area, if the robot device 100 tilts forward, light emitted by the first LiDAR sensor 111 is reflected on the ground and then detected, so the first distance from an obstacle (e.g., the ground) identified based on the first sensing data received from the first LiDAR sensor 111 can rapidly decrease.

[0086] For example, a change amount of the first distance from an obstacle located at a front side of the robot device 100 can not decrease in correspondence with a moving speed of the robot device 100, but the change amount of the first distance can exceed the moving speed of the robot device 100.

[0087] For example, when the robot device 100 travels in a flat area, light emitted by the first LiDAR sensor 111 is reflected on a wall located at a front side of the robot device 100 (e.g., scenario 300), and when the robot device 100 travels in a non-flat area, if the robot device 100 tilts forward, light emitted by the first LiDAR sensor 111 is reflected on the ground located at a front side of the robot device 100 (e.g., scenario 310), so the first distance from an obstacle identified based on the first sensing data received from the first LiDAR sensor 111 can rapidly decrease.

[0088] As shown in Figure 3 When the robot device 100 travels in a non-flat area, if the robot device 100 tilts backward, light emitted by the second LiDAR sensor 112 is reflected on the ground and then detected, so the second distance from an obstacle (e.g., the ground) identified based on the second sensing data received from the second LiDAR sensor 112 can rapidly decrease.

[0089] For example, the amount of change in the second distance from the obstacle located at the rear side of the robot device 100 can not increase as the moving speed of the robot device 100 increases, but the amount of change in the second distance can exceed the moving speed of the robot device 100.

[0090] For example, when the robot device 100 travels in a flat area, light emitted by the second LiDAR sensor 112 is reflected on a wall located at the rear side of the robot device 100, and when the robot device 100 travels in a non-flat area, if the robot device 100 is tilted backward, light emitted by the second LiDAR sensor 112 is reflected on the ground located at the rear side of the robot device 100, and thus the second distance from the obstacle identified based on the second sensing data received from the second LiDAR sensor 112 can rapidly decrease.

[0091] According to an embodiment, when the robot device 100 travels, if a rapid increase or a rapid decrease in the first distance is identified based on the first sensing data received from the first LiDAR sensor 111 or a rapid increase or a rapid decrease in the second distance is identified based on the second sensing data received from the second LiDAR sensor 112, the at least one processor 133 can identify a region in which the robot device 100 is located as a non-flat region.

[0092] The at least one processor 133 can obtain information about the non-flat region including at least one of a location, a size, or a shape of a region identified as the non-flat region, with the map information obtained.

[0093] According to an embodiment, the at least one processor 133 can predict a location of the robot device 100 at a time point at which the non-flat region is identified. Subsequently, the at least one processor 133 can adjust the predicted location to an actual location of the robot device 100 based on the first sensing data received from the first LiDAR sensor 111, the second sensing data received from the second LiDAR sensor 112, and the image received from the camera 113. Then, the at least one processor 133 can map a region on the map corresponding to the adjusted location to the non-flat region.

[0094] According to an embodiment, in a case where map information of a space in which the robot device 100 is located is obtained according to a SLAM operation, the at least one processor 133 can identify at least one of a location, a size, or a shape of a non-flat region within the space by identifying (or dividing, extracting) a region mapped to the non-flat region.

[0095] Further, the at least one processor 133 can predict a position of the robot device 100 at a time point at which the flat area is identified. Subsequently, the at least one processor 133 can adjust the predicted position to an actual position of the robot device 100 based on the first sensing data received from the first LiDAR sensor 111, the second sensing data received from the second LiDAR sensor 112, and the image received from the camera 113. Subsequently, the at least one processor 133 can map an area on the map corresponding to the adjusted position to the flat area.

[0096] According to an embodiment, in a case where map information of a space in which the robot device 100 is located is obtained according to a SLAM operation, the at least one processor 133 can identify at least one of a position, a size, or a shape of a flat area within the space by identifying (or dividing, extracting) an area mapped to the flat area.

[0097] Figure 4 FIG. 400 is a diagram illustrating first sensing data and second sensing data according to an embodiment of the disclosure.

[0098] Referring to Figure 3 the scene 300 of FIG. 3A and Figure 4 the t0-t1 time period in the diagram 400 of FIG. 4A, if the robot device 100 travels in the flat area, a first distance (A) from a wall located at a front side of the robot device 100 can decrease to correspond to a moving speed of the robot device 100.

[0099] Further, if the robot device 100 moves in the flat area, a second distance (B) from a wall located at a rear side of the robot device 100 can increase to correspond to the moving speed of the robot device 100, as shown in the t0-t1 time period in the diagram 450 of FIG. 4B. Figure 4

[0100] According to an embodiment, if the first distance (A) decreases to correspond to the moving speed of the robot device 100, or the second distance (B) increases to correspond to the moving speed of the robot device 100, the at least one processor 133 can identify an area in which the robot device 100 moves as the flat area.

[0101] ​According to an embodiment, if the obstacle is not located at the front side of the robot device 100, the at least one processor 133 can identify the first distance (A) as "0" or infinity. For example, if the obstacle is not located at the front side of the robot device 100, the light emitted by the first LiDAR sensor 111 is not reflected on the obstacle, and thus, if the light emitted by the first LiDAR sensor 111 is not returned within a threshold time or more, the at least one processor 133 can identify the first distance (A) as "0" or infinity. According to an embodiment, if the first distance (A) is identified as "0" or infinity while the robot device 100 travels, the at least one processor 133 can identify the area in which the robot device 100 moves as a flat area.

[0102] According to an embodiment, if the obstacle is not located at the rear side of the robot device 100, the at least one processor 133 can identify the second distance (B) as "0" or infinity. Also, according to an embodiment, when the robot device 100 travels, if the second distance (B) is identified as "0" or infinity, the at least one processor 133 can identify the area in which the robot device 100 moves as a flat area.

[0103] According to an embodiment, due to the specifications of the LiDAR sensor 110, for example, if no obstacle is detected within the maximum distance that the LiDAR sensor 110 can detect, the at least one processor 133 can not identify the area in which the robot device 100 is located as a flat area, but identify the area as a flat area or a non-flat area based on sensing data of another sensor (e.g., an inertial measurement unit (IMU), a camera 113, etc.) other than the LiDAR sensor 110.

[0104] Accordingly, the at least one processor 133 can explicitly identify a non-flat area based on sensing data received from at least one sensor of a plurality of sensors such as the LiDAR sensor 110, the camera 113, an inertial measurement unit (IMU), etc.

[0105] Referring to Figure 3 the scenario 310 of FIG. 3A and Figure 4 the t1-t2 period in the graph 400 of FIG. 4, if the robot device 100 travels in a non-flat area, in particular, if the robot device 100 is tilted forward, the first distance (A) to the obstacle (e.g., the ground) located at the front side of the robot device 100 can rapidly decrease. Accordingly, as shown in the t1-t2 period in the graph 450 of FIG. 4, the second distance (B) to the obstacle (e.g., the wall) located at the rear side of the robot device 100 can rapidly increase. Figure 4

[0106] Referring to Figure 3 the scenario 320 of FIG. 3B and​Figure 4 If the robot device 100 traveling in the non-flat area is tilted backward in the t1-t2 period of the graph 400, the obstacle located at the front side of the robot device 100 changes to a wall instead of the ground, and thus the first distance A) from the obstacle can rapidly increase. Also, as the obstacle located at the rear side of the robot device 100 changes to the ground instead of the wall, the second distance (B) from the obstacle can rapidly decrease.

[0107] According to an embodiment, as Figure 4 If the rapid increase or decrease in the distance (e.g., the first distance (A) or the second distance (B)) from the obstacle is repeatedly detected based on at least one of the first sensing data or the second sensing data, as shown in the t1-t2 period of the graphs 400 and 450, the at least one processor 133 can identify the area in which the robot device 100 moves as the non-flat area.

[0108] In various embodiments of the disclosure, the rapid decrease and the rapid increase can include a change in distance (decrease in distance, increase in distance) that exceeds a unit time multiplied by a moving speed of the robot device 100.

[0109] For example, if the increase and decrease in the first distance are repeated, or the increase and decrease in the second distance are repeated, during a predetermined time (e.g., 10 seconds), and the amount of change in distance exceeds the predetermined time multiplied by the moving speed of the robot device 100, the at least one processor 133 can identify the area in which the robot device 100 moves as the non-flat area.

[0110] Here, the predetermined time can be inversely proportional to the moving speed of the robot device 100. For example, as the moving speed of the robot device 100 moving in the non-flat area is faster, the increase and decrease in distance between the LiDAR sensor 110 and the obstacle are repeated faster, and thus the at least one processor 133 can identify whether the area in which the robot device 100 moves is the non-flat area by identifying the increase and decrease in distance during a relatively shorter time.

[0111] For example, as the moving speed of the robot device 100 moving in the non-flat area is slower, the increase and decrease in distance between the LiDAR sensor 110 and the ground are repeated slower, and thus the at least one processor 133 can identify whether the area in which the robot device 100 moves is the non-flat area by identifying the increase and decrease in distance during a relatively longer time.

[0112] Accordingly, as Figure 4As shown in the time period t1-t2 in Figures 400 and 450, if the increasing-decreasing pattern or fluctuation (or the rate of change of distance) of the distance from the ground detected in the first direction does not correspond to the moving speed of the robot device 100, or the increasing-decreasing pattern or fluctuation of the distance from the ground detected in the second direction does not correspond to the moving speed of the robot device 100, then at least one processor 133 can identify the area where the robot device 100 is located in the time period t1-t2 as a non-flat area.

[0113] In the foregoing embodiments, it is assumed that at least one processor 133 identifies the area where the robot device 100 is located as a non-flat region based on changes in distance (e.g., a pattern or fluctuation in distance increase or decrease, the rate of change in distance, etc.). However, this disclosure is not limited thereto, and at least one processor 133 may consider changes in distance from obstacles when identifying the area where the robot device 100 is located as a non-flat region.

[0114] For example, such as Figure 4 As shown in the time period t1-t2 in Figures 400 and 450, if the increasing-decreasing pattern or fluctuation (or the rate of change of distance) of the detected distance to the obstacle (e.g., a wall or device) in the first direction does not increase or decrease to correspond to the moving speed of the robot device 100 (or, does not increase or decrease regularly according to the movement of the robot device 100), or if the increasing-decreasing pattern or fluctuation of the detected distance to the obstacle in the second direction does not increase or decrease to correspond to the moving speed of the robot device 100, then at least one processor 133 may identify the area where the robot device 100 is located as a non-flat area during the time period t1-t2.

[0115] According to an embodiment, at least one processor 133 can identify the area where the robot device 100 moves as a flat area or a non-flat area based on at least one of the first sensing data or the second sensing data, and obtain ground information that the space is divided into flat areas or non-flat areas.

[0116] According to an embodiment, the ground information may include multiple subspaces, and each of the multiple subspaces may correspond to a flat area and a non-flat area.

[0117] Figure 5 This is a diagram illustrating a depth image according to an embodiment of the present disclosure.

[0118] The robotic device 100 according to an embodiment may include a depth camera.

[0119] Depth cameras may include time-of-flight (ToF) camera sensors.

[0120] The ToF camera sensor can irradiate a signal (e.g., near-infrared rays, ultrasonic waves, laser, etc.) and receive a reflected signal when the irradiated signal is reflected by an object. The at least one processor 133 can measure a distance (or depth) between the ToF camera sensor and the object by measuring a time taken for the ToF camera sensor to receive the reflected signal. According to an embodiment, the at least one processor 133 can obtain depth information of the object based on the distance between the ToF camera sensor and the object.

[0121] The ToF camera sensor is merely an example, and the disclosure is not limited thereto, and the robot device 100 can obviously include a radar sensor, an ultrasonic sensor, an infrared sensor, etc.

[0122] According to an embodiment, the at least one processor 133 can control the depth camera to capture a floor corresponding to a position of the robot device 100.

[0123] For example, the depth camera can irradiate a signal in a first direction corresponding to a front side of the robot device 100 and receive a signal reflected from an object (e.g., a floor).

[0124] According to an embodiment, the at least one processor 133 can obtain a plurality of depth images by controlling the depth camera to capture an object at predetermined time intervals.

[0125] According to an embodiment, for each of the plurality of depth images, a composite depth image indicating a height difference of the object can be obtained.

[0126] For example, the at least one processor 133 can identify a region (i.e., a non-flat region) that is not flat due to a height difference of the floor based on the plurality of depth images.

[0127] For example, the at least one processor 133 can obtain a composite depth image based on the plurality of depth images, and the composite depth image can be a 3D image indicating a height difference (or a height variation) of the floor by performing 3D modeling of the floor.

[0128] According to an embodiment, the at least one processor 133 can identify a plurality of sub-spaces by dividing a space into a flat region and a non-flat region based on first sensing data of the first LiDAR sensor 111 and second sensing data of the second LiDAR sensor 112.

[0129] For example, the at least one processor 133 can divide the space into a plurality of sub-spaces by identifying independent regions surrounded by walls in the space. The at least one processor 133 can also divide the space into a plurality of sub-spaces by identifying furniture, steps, door thresholds, etc. in the space.

[0130] According to an embodiment, the at least one processor 133 can divide each of the plurality of subspaces into a flat region and a non-flat region. However, the present disclosure is not limited thereto, and the at least one processor 133 can divide a portion of any one of the subspaces into a non-flat region and the remaining portion into a flat region.

[0131] According to an embodiment, the at least one processor 133 can identify a change in height of the ground corresponding to the non-flat region of the subspaces (or a portion of the subspaces) of the plurality of subspaces based on the plurality of depth images received from the depth camera. Also, according to an embodiment, the at least one processor 133 can control the travel of the robot device 100 based on the identified change in height.

[0132] According to an embodiment, the at least one processor 133 can also divide the space into a plurality of subspaces based on the plurality of depth images received from the depth camera.

[0133] For example, in the case where the map information is obtained, the at least one processor 133 can obtain a 3D image indicating a change in height of the ground inside the space, and obtain information related to the non-flat region indicating a region in which the change in height is detected (or a region in which the change in height is not included in a threshold range) as the non-flat region, and obtain information related to the flat region indicating a region in which the change in height is not detected (or a region in which the change in height is included in a threshold range) as the flat region.

[0134] As described above, the at least one processor 133 can obtain the ground information including information related to the flat region inside the space and information related to the non-flat region inside the space.

[0135] Figure 6a FIG. 1 is a diagram illustrating a change in pitch of a robot device based on an inertial measurement unit according to an embodiment of the present disclosure.

[0136] The robot device 100 according to an embodiment can include an inertial measurement unit (IMU).

[0137] The inertial measurement unit (hereinafter, IMU) according to an embodiment can include at least one of a gyro sensor, an accelerometer sensor, or a magnetometer or a compass sensor. The robot device 100 can identify whether the robot device 100 is tilted, a direction of tilt of the robot device 100, a degree of tilt, etc., based on sensing values of various inertial measurement units.

[0138] According to an embodiment, the IMU can obtain each of roll, pitch, and yaw indicating a posture of the robot device 100 as the third sensing data. Here, the roll can indicate a tilt of the robot device 100 in a left-right direction (e.g., a rotation angle based on an x-axis (a longitudinal axis)), the pitch can indicate a tilt of the robot device 100 in a front-back (forward and backward) direction (e.g., a rotation angle based on a y-axis (a horizontal axis)), and the yaw can indicate a tilt of the robot device 100 in a z-axis direction (e.g., a rotation angle based on a z-axis (a vertical axis)).

[0139] According to an embodiment, the at least one processor 133 can detect a posture of the robot device 100 based on the third sensing data received from the IMU.

[0140] Figure 6a is a side view of the robot device 100, and a pitch of the robot device 100 can change when the robot device 100 moves in a non-flat area.

[0141] For example, if the robot device 100 is tilted forward, the pitch according to the third sensing data received from the IMU can be a positive (+) value, and if the robot device 100 is tilted backward, the pitch according to the third sensing data received from the IMU can be a negative (-) value. It should be understood that the disclosure is not limited to this embodiment.

[0142] The at least one processor 133 according to an embodiment can detect a posture of the robot device 100 based on the third sensing data, and if the detected posture exceeds a threshold angle (e.g., if the pitch of the robot device 100 exceeds a threshold), the at least one processor 133 can identify an area in which the robot device 100 moves when the posture of the robot device 100 is tilted forward or backward as a non-flat area.

[0143] Figure 6b is a graph illustrating a change in roll of a robot device based on an inertial measurement unit according to an embodiment of the disclosure.

[0144] Figure 6b The graph of the upper left side of includes a plan view showing the robot device 100 and a wall from above.

[0145] When the robot device 100 moves in a non-flat area, a roll of the robot device 100 can change.

[0146] For example, if the robot device 100 is tilted to the right side, the roll according to the third sensing data received from the IMU can be a positive (+) value, and if the robot device 100 is tilted to the left side, the roll according to the third sensing data received from the IMU can be a negative (-) value. It should be understood that the disclosure is not limited to this embodiment.

[0147] The at least one processor 133 according to an embodiment can detect a posture of the robot device 100 based on the third sensing data, and if the detected posture exceeds a threshold angle (for example, if the roll of the robot device 100 exceeds a threshold), the at least one processor 133 can identify a region in which the robot device 100 moves when the posture of the robot device 100 is tilted to the right or to the left as a non-flat region.

[0148] According to an embodiment, the at least one processor 133 can identify a region in which the robot device 100 moves as a flat region or a non-flat region based on at least one of sensing data (for example, first sensing data and second sensing data) of the LiDAR sensor 110 or third sensing data of the IMU.

[0149] Figure 7 is a diagram for illustrating a travel level according to an embodiment of the disclosure.

[0150] The at least one processor 133 according to an embodiment can control the robot device 100 to travel at any one of a plurality of travel levels.

[0151] For example, the plurality of travel levels can be divided based on a travel speed of the robot device 100. Each of the plurality of travel levels according to an embodiment of the disclosure can include information for controlling travel of the robot device 100.

[0152] For example, the information for controlling travel can include a travel speed range (for example, a minimum speed and a maximum speed) of the robot device 100, an average travel speed of the robot device 100, etc.

[0153] For example, a first travel level of the plurality of travel levels can include high-speed travel of the robot device 100, a second travel level can include regular travel of the robot device 100, a third travel level can include low-speed travel of the robot device 100, and a fourth travel level can include evasive travel of the robot device 100. However, this is merely an example for convenience of explanation, and the disclosure is not limited thereto. For example, the first travel level can include high-speed travel of the robot device 100, and the second travel level can include low-speed travel of the robot device 100.

[0154] However, the disclosure is not limited thereto, and the information for controlling travel can include an operation mode of the robot device 100. For example, a first travel level of the plurality of travel levels can include a power mode (e.g., a high output mode) of the robot device 100, a second travel level can include a regular mode of the robot device 100, and a third travel level can include a low power mode (e.g., an echo mode) of the robot device 100. According to an embodiment, each of the plurality of travel levels can include information for controlling a speed, a radius of rotation, and an output (a number of rotations of a motor) of a motor provided on the robot device 100 in relation to an operation of a gripper (e.g., a robot hand as a distal end of a joint type robot) provided on the robot device 100.

[0155] The at least one processor 133 according to an embodiment can cause the robot device 100 to operate (or travel) at any one travel level of the plurality of travel levels based on sensing data (e.g., first sensing data, second sensing data, a depth image, third sensing data, etc.) received from a sensor (e.g., a LiDAR sensor 110, a depth camera, an IMU, etc.) disposed on the robot device 100. For example, the at least one processor 133 can identify a travel level corresponding to a current position of the robot device 100 among the plurality of travel levels based on the sensing data received from the sensor in real time.

[0156] According to an embodiment, the at least one processor 133 can obtain a travel route (or a movement route) of the robot device 100 by identifying a travel level corresponding to each of a plurality of sub-spaces in a space based on map information.

[0157] For example, if a first sub-space of the plurality of sub-spaces corresponds to a flat area based on the map information, the at least one processor 133 can identify a travel level of the first sub-space as a first travel level of the plurality of travel levels, and if a second sub-space corresponds to a non-flat area, the at least one processor 133 can identify a travel level of the second sub-space as a second travel level of the plurality of travel levels. According to an embodiment, a travel speed of the robot device 100 according to the second travel level can be relatively slower than a travel speed according to the first travel level.

[0158] According to an embodiment, the at least one processor 133 can obtain a travel route in which the robot device 100 travels at a first travel level when moving in a first sub-space and travels at a second travel level when moving in a second sub-space.

[0159] For example, if the robot device 100 travels in a non-flat area, there is a risk that a threat can occur to an object (e.g., a person, an obstacle, etc.) adjacent to the robot device 100, and thus the at least one processor 133 can cause the robot device 100 to operate at a travel level (e.g., low-speed travel) corresponding to the non-flat area, based on the map information, while the robot device 100 travels in the non-flat area.

[0160] However, the disclosure is not limited thereto, and the at least one processor 133 can identify the travel level of the second subspace as an evasive travel among the plurality of travel levels.

[0161] For example, if the robot device 100 travels in a non-flat area, there is a risk that an accident can occur, and thus the at least one processor 133 can control the robot device 100 so that the robot device 100 travels in an adjacent area (e.g., a flat area adjacent to the non-flat area) of the non-flat area and does not travel in the non-flat area. Accordingly, the at least one processor 133 can identify the travel level of the non-flat area as an evasive travel based on the map information, and set a movement route of the robot device 100 so that the robot device 100 does not travel in the non-flat area.

[0162] According to an embodiment, the travel level information can include a travel level corresponding to each of the plurality of subspaces, and the at least one processor 133 can control travel of the robot device 100 based on a travel level of a subspace corresponding to a location of the robot device 100 among the plurality of subspaces obtained from the travel level information.

[0163] Figure 8 is a diagram for illustrating an object loaded on a robot device according to an embodiment of the disclosure.

[0164] According to an embodiment, an object can be loaded on the robot device 100, and the robot device 100 can move the loaded object.

[0165] According to an embodiment, the robot device 100 can include a weight detection sensor, and the at least one processor 133 can detect a weight of an object loaded on the robot device 100 through the weight detection sensor.

[0166] According to an embodiment, the robot device 100 can identify any one travel level among the plurality of travel levels according to the weight of the loaded object.

[0167] For example, if the weight of the loaded object is less than a threshold value, or there is no loaded object, the at least one processor 133 can cause the robot device 100 to operate at a first travel level among the plurality of travel levels.

[0168] For example, if there is no object loaded on the robot device 100, there is no risk of an accident occurring due to a loaded object falling while the robot device 100 is traveling, and thus the at least one processor 133 can control the robot device 100 to travel at a first travel level (e.g., high-speed travel, regular travel, etc.).

[0169] For example, if there is no object loaded on the robot device 100, there is no risk of an object loaded on the robot device 100 falling even if the robot device 100 travels in a non-flat area, and thus the at least one processor 133 can control the robot device 100 to travel at a first travel level in both flat areas and non-flat areas.

[0170] For example, if the weight of the loaded object is greater than or equal to a threshold value, the at least one processor 133 can cause the robot device 100 to operate at a second travel level among a plurality of travel levels.

[0171] According to an embodiment, if the weight of the object loaded on the robot device 100 is greater than or equal to a threshold value, the at least one processor 133 can control the robot device 100 to travel at a first travel level in a first subspace corresponding to a flat area among a plurality of subspaces, and control the robot device 100 to travel at a second travel level in a second subspace corresponding to a non-flat area.

[0172] For example, there is a risk that an object loaded on the robot device 100 can fall due to vibrations generated when the robot device 100 travels in a non-flat area, and thus the at least one processor 133 can control the robot device 100 to travel at a second travel level (e.g., low-speed travel) in the non-flat area.

[0173] However, the disclosure is not limited thereto, and if the weight of the object loaded on the robot device 100 is greater than or equal to a threshold value, there is a risk that the object loaded on the robot device 100 can fall even if the robot device 100 travels in a flat area, and thus the at least one processor 133 can control the robot device 100 to travel at a second travel level in both flat areas and non-flat areas.

[0174] According to an embodiment, if the weight of the loaded object is greater than or equal to a threshold value, the at least one processor 133 can set a non-flat area to a fourth travel level (e.g., evasive travel) so that the robot device 100 does not travel in the non-flat area.

[0175] For example, if the weight of the loaded object is greater than or equal to a threshold value, there is a risk that a malfunction can be caused in the driver 120 of the robot device 100, or if the robot device 100 travels in a non-flat area, the object can fall, and thus the robot device 100 can not travel in the non-flat area but travel in an adjacent flat area.

[0176] For example, if the weight of the loaded object is greater than or equal to a threshold value, and the change in height of the non-flat area exceeds a threshold range (for example, if the change in height exceeds 5 cm), the at least one processor 133 can set the non-flat area to the fourth travel level (for example, evasive travel) so that the robot device 100 does not travel in the non-flat area, and thus the robot device 100 can not travel in the non-flat area but travel in an adjacent flat area.

[0177] Figure 9 FIG. 1 is a diagram illustrating an evasive travel of a robot device according to an embodiment of the disclosure.

[0178] According to an embodiment, the at least one processor 133 can obtain a travel route (or a movement route) in which the robot device 100 travels in a space based on the map information.

[0179] For example, the at least one processor 133 can identify each of the plurality of subspaces as a flat area or a non-flat area.

[0180] According to an embodiment, the at least one processor 133 can obtain a movement route in which, if a first subspace of the plurality of subspaces corresponds to a flat area, the robot device 100 travels in the first subspace at a first travel level of a plurality of travel levels, and if a second subspace corresponds to a non-flat area, the robot device 100 travels in the second subspace at a second travel level.

[0181] According to an embodiment, the at least one processor 133 can adjust the movement route by using a change in height of the ground identified based on a plurality of depth images received from a depth camera.

[0182] For example, the at least one processor 133 can obtain a movement route in which the robot device 100 travels to avoid a non-flat area in which a change in height of the ground is greater than or equal to a threshold height.

[0183] For example, in a non-flat area in which the height variation is greater than or equal to a threshold height, it can be impossible to move (or, it can be impossible to climb) by the driver provided in the robot device 100, or there is a risk that the object loaded on the robot device 100 can fall. Accordingly, according to an embodiment, the at least one processor 133 can set a movement route so that the robot device 100 travels to avoid the non-flat area in which the variation in height is greater than or equal to the threshold height.

[0184] According to an embodiment, the at least one processor 133 can adjust the movement route of the robot device 100 using the posture of the robot device 100 identified based on the third sensing data received from the IMU.

[0185] For example, if at least one of the rotation angle based on the x-axis (longitudinal axis) of the robot device 100, the rotation angle based on the y-axis (horizontal axis), or the rotation angle based on the z-axis (vertical axis) of the robot device 100 exceeds a threshold angle, there is a risk that the object loaded on the robot device 100 can fall, and thus the at least one processor 133 can set a movement route so that the robot device 100 performs evasive travel.

[0186] Returning Figure 2 The robot device 100 according to an embodiment of the disclosure can include a camera.

[0187] According to an embodiment, the camera can capture the ground corresponding to the position of the robot device 100, and the at least one processor 133 can obtain material information of the ground by analyzing an image received from the camera.

[0188] According to an embodiment, the at least one processor 133 can control the travel of the robot device 100 based on the material information.

[0189] For example, the at least one processor 133 can obtain a plurality of images by capturing the ground at predetermined time intervals via the camera, and obtain material information corresponding to the ground of the space by analyzing each of the plurality of images.

[0190] For example, the material of the ground can be divided into a material having a large coefficient of friction and a material having a small coefficient of friction. For example, the ground can include various materials such as a carpet, a tile, a sheet, a linoleum, a solid wood, an epoxy, a polyurethane, a marble, a PVC material floor, etc., and the coefficient of friction varies according to the material constituting the ground.

[0191] The material of the ground can be determined according to the material (or substance) of the ground for each of the plurality of sub-spaces. For example, the material of the ground of the sub-space can be implemented as equal to.

[0192] According to an embodiment, the at least one processor 133 can identify a material of the floor by analyzing an image of the floor captured through the camera, and if the identified material is a slippery material (e.g., a material having a coefficient of friction less than or equal to a threshold value), the at least one processor 133 can adjust a travel level for travel of the robot device 100.

[0193] For example, if a first sub-space among the plurality of sub-spaces corresponds to a flat area, the at least one processor 133 can control the robot device 100 to travel at a first travel level when traveling in the first sub-space.

[0194] According to an embodiment, if a material of the floor constituting the first sub-space is a slippery material having a coefficient of friction less than or equal to a threshold value, the robot device 100 can deviate from a movement route (e.g., slippage of the robot device 100) when traveling in the first sub-space, or there is a risk that an object loaded on the robot device 100 can fall, and thus the at least one processor 133 can control the robot device 100 to travel at a second travel level when moving in the first sub-space, or adjust the movement route so that the robot device 100 travels to avoid the first sub-space. For example, a movement speed of the robot device 100 according to the second travel level can be slower than a movement speed of the robot device 100 according to the first travel level.

[0195] According to an embodiment, the at least one processor 133 can obtain material information of the floor by inputting an image of the floor captured via the camera into a neural network model.

[0196] The neural network model according to an embodiment can be a model trained to output material information of a floor in a case where an image of the floor is input, by using a plurality of sample images of floors capturing various materials as training data.

[0197] The functions related to artificial intelligence according to the present disclosure are operated by the at least one processor 133 and the memory of the robot device 100.

[0198] Here, the at least one processor 133 can include at least one of a central processing unit (CPU), a graphics processing unit (GPU), or a neural processing unit (NPU), but is not limited to the aforementioned examples of the processor.

[0199] The CPU is a general-purpose processor that can not only perform general operations but also perform artificial intelligence operations, and it can efficiently perform a complex program through a multi-tier cache structure. The CPU is advantageous for a serial processing method that can achieve systematic linkage between a previous calculation result and a next calculation result through sequential calculation. In addition, the general-purpose processor is not limited to the aforementioned example, except for a case where the general-purpose processor is designated as the aforementioned CPU.

[0200] The GPU is a processor for large-scale operations such as floating-point operations for graphics processing, and it can perform large-scale operations in parallel through a large-scale integration of cores. In particular, compared to the CPU, the GPU can be advantageous for a parallel processing method such as a convolution operation. Also, the GPU can function as a co-processor for supplementing the functions of the CPU. Meanwhile, the processor for large-scale operations is not limited to the above-described example, except for the case where it is designated as the above-described GPU.

[0201] The NPU is a processor dedicated to artificial intelligence operations using artificial neural networks, and it can implement each layer constituting an artificial neural network as hardware (e.g., silicon). Here, the NPU is designed to be specialized according to the specifications required by a company, and thus it has a lower degree of freedom compared to the CPU or the GPU, but it can efficiently process artificial intelligence operations required by a company. Also, as a processor dedicated to artificial intelligence operations, the NPU can be implemented in various forms such as a tensor processing unit (TPU), an intelligent processing unit (IPU), a visual processing unit (VPU), etc. Meanwhile, the artificial intelligence processor is not limited to the above-described example, except for the case where it is designated as the above-described NPU.

[0202] Also, the one or more processors can be implemented as a system on chip (SoC). Here, in the SoC, in addition to the one or more processors, a memory and a network interface such as a bus for data communication between the processor and the memory, etc. can be included.

[0203] When a plurality of processors are included in the system on chip (SoC) included in the robot device 100, the robot device 100 can perform an operation related to artificial intelligence (e.g., an operation related to learning or inference of an artificial intelligence model) by using some of the plurality of processors. For example, the robot device 100 can perform an operation related to artificial intelligence by using at least one of a GPU, an NPU, a VPU, a TPU, or a hardware accelerator dedicated to an artificial intelligence operation such as a convolution operation, a matrix multiplication operation, etc. among the plurality of processors. However, this is merely an example, and the robot device 100 can obviously process an operation related to artificial intelligence by using a general-purpose processor such as a CPU, etc.

[0204] Also, the robot device 100 can perform an operation for a function related to artificial intelligence by using a multi-core (e.g., a dual-core, a quad-core, etc.) included in the at least one processor 133. Specifically, the robot device 100 can perform an artificial intelligence operation such as a convolution operation, a matrix multiplication operation, etc. in parallel by using a multi-core included in the at least one processor 133.

[0205] The one or more processors 133 perform control to process input data according to a pre-defined operation rule or an artificial intelligence model stored in the memory. The pre-defined operation rule or the artificial intelligence model is characterized in that it is made by learning.

[0206] Here, made by learning means that the pre-defined operation rule or the artificial intelligence model having a desired characteristic is made by applying a learning algorithm to a plurality of training data. Such learning can be performed in the device that performs artificial intelligence according to the disclosure itself, or can be performed by a separate server / system.

[0207] The artificial intelligence model can be composed of a plurality of neural network layers. At least one layer has at least one weight value, and an operation of the at least one layer is performed by an operation result of a previous layer and at least one defined operation. As examples of the neural network, there are a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, and a Transformer, but the neural network in the disclosure is not limited to the above examples, except for explicitly designated cases.

[0208] The learning algorithm is a method of training a specific object device (e.g., a robot) by using a plurality of training data and thereby causing the specific object device to make a decision or make a prediction on its own. As examples of the learning algorithm, there are supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but the learning algorithm in the disclosure is not limited to the above examples, except for specific cases.

[0209] Figure 10 is a flowchart for illustrating a control method of a robot device according to an embodiment of the disclosure.

[0210] Referring to Figure 10 In the control method of the robot device for obtaining map information, in operation S1010, information about a floor of a space in which the robot device is located is obtained based on first sensing data received from a first LiDAR sensor and second sensing data received from a second LiDAR sensor.

[0211] In operation S1020, a plurality of sub-spaces included in the space are identified based on the floor information.

[0212] In operation S1030, travel level information including a travel level corresponding to each of the plurality of sub-spaces is obtained.

[0213] The floor information according to the embodiment can include information about a flat area and a non-flat area in the space, and in the operation S1020 of identifying a plurality of sub-spaces, the plurality of sub-spaces can be identified by dividing the space into the flat area or the non-flat area based on the floor information.

[0214] The operation S1030 of obtaining the travel level information includes the following operations: identifying a travel level of a first sub-space as a first travel level based on the floor of the first sub-space corresponding to the flat area, and identifying a travel level of a second sub-space as a second travel level based on the floor of the second sub-space corresponding to the non-flat area, and the travel speed of the robot device according to the second level can be relatively slower than the travel speed of the robot device according to the first level.

[0215] According to the embodiment, the operation S1020 of identifying a plurality of sub-spaces can include the step of identifying an area in which the robot device is located as a non-flat area based on the distance between the first LiDAR sensor included in the first sensing data and the floor on which the robot device is located repeatedly increasing and decreasing and the distance between the second LiDAR sensor included in the second sensing data and the floor repeatedly increasing and decreasing.

[0216] According to the embodiment, the operation of identifying an area in which the robot device is located as a non-flat area can include the step of identifying an area in which the robot device is located as a non-flat area based on the first sensing data and the second sensing data if a pattern of increase and decrease of the distance from the floor or fluctuation of the distance detected in a first direction or a pattern of increase and decrease of the distance from the floor or fluctuation of the distance detected in a second direction does not correspond to the moving speed of the robot device in a first time period.

[0217] The control method according to the embodiment of the disclosure can include the operations of obtaining a plurality of depth images by controlling a depth camera to capture a floor corresponding to a position of a robot device at a predetermined time interval, and identifying a plurality of sub-spaces included in a space based on the plurality of depth images.

[0218] The control method according to the embodiment of the disclosure can further include the step of detecting a posture of the robot device based on third sensing data received from an inertial measurement unit (IMU), and the operation S1020 of identifying a plurality of sub-spaces can include the step of identifying a plurality of sub-spaces based on the posture of the robot device and floor information, and the third sensing data can include at least one of roll, pitch, or yaw indicating the posture of the robot device.

[0219] The operation S1020 of identifying the plurality of subspaces can include the following operation: based on at least one of the roll, the pitch, or the yaw being greater than or equal to a threshold angle, identifying a posture of the robot device as tilted, and identifying an area in which the robot device is located as identified by the tilt as a non-flat area.

[0220] The control method according to an embodiment of the disclosure can include the following operations: obtaining an image by controlling a camera to capture a ground surface corresponding to a position of a robot device, obtaining material information of the ground surface based on the image, and obtaining travel level information including a travel level corresponding to each of a plurality of subspaces based on the material information.

[0221] Figure 11 is a flowchart illustrating a method of controlling travel of a robot device according to an embodiment of the disclosure.

[0222] Referring to Figure 11 In the control method of a robot device for controlling travel of a robot device, in operation S1110, a travel level of a subspace corresponding to a position of the robot device among a plurality of subspaces is obtained from travel level information.

[0223] In operation S1110, travel of the robot device is controlled based on the obtained travel level.

[0224] The control method according to an embodiment of the disclosure can further include the following operations: obtaining a plurality of depth images by controlling a depth camera to capture a ground surface corresponding to a position of a robot device at a predetermined time interval; identifying a height variation of the ground surface of a subspace corresponding to a non-flat area among a plurality of subspaces based on the plurality of depth images; and controlling travel of the robot device based on the identified height variation.

[0225] The control method according to an embodiment of the disclosure can further include the following operations: identifying a posture of the robot device based on third sensing data received from an inertial measurement unit (IMU), and controlling travel of the robot device based on the posture of the robot device.

[0226] The control method according to an embodiment of the disclosure can further include the following operations: obtaining an image by controlling a camera to capture a ground surface corresponding to a position of a robot device; obtaining material information of the ground surface based on the image; and controlling travel of the robot device based on the material information.

[0227] The control method according to an embodiment of the disclosure can include operations of detecting a weight of an object loaded on a robot device, and based on the detected weight being greater than or equal to a threshold, controlling travel of the robot device by identifying a travel level corresponding to a sub-space corresponding to a position of the robot device among a plurality of sub-spaces, and based on the detected weight being less than the threshold, controlling the travel of the robot device without changing the travel level.

[0228] Various embodiments of the disclosure can be applied not only to a robot device but also to various types of electronic devices.

[0229] In addition, the above-described various embodiments of the disclosure can be implemented in a recording medium that can be read by a computer or a device similar thereto by using software, hardware, or a combination thereof. In some cases, the embodiments described in the specification can be implemented as a processor itself. According to implementation by software, embodiments such as processes and functions described in the specification can be implemented as separate software modules. Each software module can perform one or more functions and operations described in the specification.

[0230] In addition, computer instructions for performing processing operations of a robot device according to the aforementioned various embodiments of the disclosure can be stored in a non-transitory computer readable medium. When such computer instructions stored in the non-transitory computer readable medium are executed by a processor, they can cause processing operations at the robot device 100 according to the aforementioned various embodiments to be performed by a specific machine.

[0231] The non-transitory computer readable medium refers to a medium that stores data semi-permanently and is readable by a machine, rather than a medium that stores data for a short time such as a register, a cache, and a memory. As a specific example of the non-transitory computer readable medium, there can be a CD, a DVD, a hard disk, a Blu-ray disk, a USB, a memory card, a ROM, etc.

[0232] Furthermore, although preferred embodiments of the disclosure have been shown and described, the disclosure is not limited to the specific embodiments described above, and it will be apparent to those skilled in the art that various modifications can be made without departing from the spirit of the disclosure claimed in the appended claims. Furthermore, it is intended that such modifications will not be interpreted independently of the technical idea or prospect of the disclosure.

Claims

1. A robot device comprising: a memory; a first LiDAR sensor configured to detect an obstacle in a first direction; a second LiDAR sensor configured to detect an obstacle in a second direction opposite to the first direction; and at least one processor configured to: obtain floor information about a floor of a space in which the robot device is located, based on first sensing data received from the first LiDAR sensor and second sensing data received from the second LiDAR sensor, identify a plurality of sub-spaces included in the space, based on the floor information, obtain travel level information and store the travel level information in the memory, wherein the travel level information includes a travel level associated with each of the plurality of sub-spaces, and control movement of the robot device based on a travel level of a sub-space corresponding to a location of the robot device, wherein the sub-space is among the plurality of sub-spaces, and wherein the travel level is obtained from the travel level information stored in the memory. 2.The robot device of claim 1, the floor information includes: wherein information about a flat area and a non-flat area in the space, and the at least one processor is further configured to: identify the plurality of sub-spaces by dividing the space into a flat area or a non-flat area based on the floor information. 3.The robot device of claim 2, the at least one processor is further configured to: wherein identify a travel level of a first sub-space among the plurality of sub-spaces as a first travel level, based on a floor of the first sub-space corresponding to a flat area, and identify a travel level of a second sub-space among the plurality of sub-spaces as a second travel level, based on a floor of the second sub-space corresponding to a non-flat area, and wherein a travel speed of the robot device at the second travel level is slower than a travel speed of the robot device at the first travel level. 4.The robot device of claim 1, the at least one processor is further configured to: wherein, identify an area in which the robot device is located as a non-flat area, based on fluctuations of a first distance and a second distance, wherein the first distance is between the first LiDAR sensor and the floor in which the robot device is located, and the second distance is between the second LiDAR sensor and the floor. 5.The robot device of claim 4, the at least one processor is further configured to: wherein, identify an area in which the robot device is located for a first time period as a non-flat area, based on the first sensing data and the second sensing data, if a first pattern of an increase and a decrease of a first distance from the floor detected in a first direction in a first time period does not correspond to a travel speed of the robot device, or a second pattern of an increase and a decrease of a second distance from the floor detected in a second direction does not correspond to the travel speed of the robot device. ​ 6.The robot device of claim 1, further comprising: a depth camera, wherein the at least one processor is further configured to: capture a ground corresponding to a location of the robot device using the depth camera to obtain a plurality of depth images at predetermined time intervals, identify a change in height of the ground of a subspace corresponding to a non-flat region among the plurality of subspaces based on the plurality of depth images, and control movement of the robot device based on the change in height. 7.The robot device of claim 1, further comprising: an inertial measurement unit (IMU), wherein the at least one processor is further configured to: detect a posture of the robot device based on third sensing data received from the IMU, and identify the plurality of subspaces based on the detected posture and the ground information, wherein the third sensing data includes: at least one of roll, pitch, or yaw indicating the posture of the robot device. 8.The robot device of claim 7, wherein wherein the at least one processor is further configured to: identify the posture of the robot device as tilted based on at least one of the roll, the pitch, or the yaw being greater than or equal to a threshold angle, and identify a region identified by the tilt as a non-flat region. 9.The robot device of claim 1, further comprising: a camera, wherein the at least one processor is further configured to: capture the ground corresponding to a location of the robot device using the camera to obtain an image, obtain material information of the ground based on the image, and control movement of the robot device based on the material information. 10.The robot device of claim 1, wherein, wherein the at least one processor is further configured to: detect a weight of an object loaded on the robot device, control movement of the robot device by identifying a driving level of a subspace corresponding to a location of the robot device based on the detected weight being greater than or equal to a threshold value, and control movement of the robot device without changing the driving level based on the detected weight being less than the threshold value. 11.A control method of an electronic device, the method comprising: receiving first sensing data in a first direction based on the robot device from a first LiDAR sensor; receiving second sensing data in a second direction opposite to the first direction from a second LiDAR sensor; obtaining ground information about a ground of a space in which the robot device is located based on the first sensing data and the second sensing data; identifying a plurality of subspaces included in the space based on the ground information; obtaining driving level information, wherein the driving level information includes a driving level associated with each of the plurality of subspaces, and controlling driving of the robot device based on a driving level of a subspace corresponding to a location of the robot device among the plurality of subspaces. 12.The control method of claim 11, wherein wherein the ground information includes: information about flat regions and non-flat regions in the space, and identifying the plurality of subspaces includes: identifying the plurality of subspaces by dividing the space into flat areas or non-flat areas based on the ground information.

13. The control method of claim 12, wherein obtaining the driving level information includes: identifying a driving level of a first subspace of the plurality of subspaces as a first driving level based on the ground of the first subspace corresponding to a flat area; and identifying a driving level of a second subspace of the plurality of subspaces as a second driving level based on the ground of the second subspace corresponding to a non-flat area, and wherein a driving speed of the robotic device at the second driving level is slower than a driving speed of the robotic device at the first driving level.

14. The control method of claim 11, wherein, identifying the plurality of subspaces includes: identifying an area in which the robotic device is located as a non-flat area based on fluctuations of a first distance and a second distance, wherein the first distance is between the first LiDAR sensor and the ground on which the robotic device is located, and the second distance is between the second LiDAR sensor and the ground.

15. The control method of claim 14, wherein, identifying an area in which the robotic device is located as a non-flat area includes: identifying an area in which the robotic device is located as a non-flat area based on the first sensing data and the second sensing data if a first pattern of increases and decreases in a first distance from the ground detected in a first direction in a first time period does not correspond to a driving speed of the robotic device, or a second pattern of increases and decreases in a second distance from the ground detected in a second direction does not correspond to the driving speed of the robotic device.

Images