Robot apparatus for identifying movement path using

Abstract

A robot apparatus is disclosed. The robot device includes: a memory; a sensor; and at least one processor that acquires, based on sensing data received from the sensor, a map corresponding to a space in which the robotic device is located, and reliability values respectively corresponding to a plurality of regions included in the map, and store the map and the plurality of reliability values in a memory. The at least one processor identifies at least one region corresponding to a reliability value equal to or higher than a threshold value based on reliability values respectively corresponding to the plurality of regions, and identifies a movement path of the robotic device in the space based on the identified at least one region.

Description

Technical Field

[0001] The present disclosure relates to a robot apparatus and a control method thereof, and more particularly, to a robot apparatus for identifying a moving path using a reliability value and a control method thereof. Background Art

[0002] Various types of robotic devices that move in a space and perform specific actions have become popular, such as service robots that replace people in shops, cafes or restaurants, robot cleaners that automatically clean the area to be cleaned by sucking foreign matter while moving on their own even without separate operation by the user, and the like.

[0003] In the related art, a robot device may obtain a map, estimate its position in the map, and move along a movement path. However, as the robot device moves, the error between the estimated position and the actual position may accumulate, resulting in significant deviation.

[0004] For example, the robot device may estimate at the end point of the moving path that the robot device itself is located at the end point in the map. However, the robot device may actually be located at a completely different location instead of the end point of the moving path.

[0005] That is, there has been a need for a method that, when the robot device moves along a movement path, identifies the degree of error between the position of a robot device estimated by the robot device itself and its actual position, and sets (or resets) the movement path in consideration of the error. Summary of the invention

[0006] According to one aspect of the present disclosure, a robot device includes: at least one memory storing at least one instruction; a sensor configured to detect the environment of the robot device and output detection data; and at least one processor configured to execute the at least one instruction to: obtain a map of a space where the robot device is located and a reliability value of each of multiple areas of the map based on the detection data received from the sensor, store the map and the reliability value of each of the multiple areas in the at least one memory, identify at least one area with a reliability value greater than or equal to a critical value based on the reliability value of each area in the multiple areas, and identify a movement path of the robot device in the space based on the at least one area.

[0007] The at least one processor of the robotic device may also be configured to execute the at least one instruction to: estimate one of the multiple areas as the position of the robotic device based on the detection data, and obtain an estimated reliability value corresponding to the estimated position of the robotic device by determining the probability that the estimated position of the robotic device and the actual position of the robotic device match each other.

[0008] The at least one processor of the robotic device may also be configured to execute the at least one instruction to: obtain movement information of the robotic device based on first detection data and second detection data continuously received from the sensor, and estimate a new position of the robotic device corresponding to the second detection data based on the movement information.

[0009] The sensor may include a light detection and ranging (LiDAR) sensor, the first detection data may include first point cloud data received from the LiDAR sensor, the second detection data may include second point cloud data received from the LiDAR sensor, and the at least one processor of the robotic device may also be configured to execute the at least one instruction to: perform an operation of combining multiple point clouds based on the first point cloud data and the second point cloud data, and based on a failure of the operation of combining the multiple point clouds, obtain an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value.

[0010] The sensor may include a camera, the first detection data may include first image data received from the camera, the second detection data may include second image data received from the camera, and the at least one processor of the robotic device may also be configured to execute the at least one instruction to: identify dynamic objects based on the first image data and the second image data, and based on the number of dynamic objects being greater than or equal to a critical number, obtain an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value.

[0011] The sensor may include a camera, the detection data may include image data received by the camera, and the at least one processor of the robotic device may also be configured to execute the at least one instruction to: identify a field of view of the camera based on the image data, and based on the angle of the field of view being less than a critical angle, obtain an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value.

[0012] The movement information may include: the movement direction of the robot device, the movement distance of the robot device or the movement speed of the robot device, and the at least one processor of the robot device may also be configured to execute the at least one instruction to: obtain an updated reliability value corresponding to the new position of the robot device based on identifying that at least one of the movement direction of the robot device, the movement distance of the robot device, and the movement speed of the robot device cannot be maintained, wherein the updated reliability value is less than the critical value.

[0013] The at least one processor of the robotic device may be further configured to execute the at least one instruction to: update a reliability value corresponding to one of the plurality of regions in the at least one memory based on the acquired reliability value.

[0014] The sensor may include a light detection and ranging (LiDAR) sensor, and the at least one processor of the robotic device may be further configured to execute the at least one instruction to: identify an error value based on an orientation of the LiDAR sensor in one of the plurality of areas based on detection data received from the LiDAR, and adjust the orientation of the LiDAR sensor based on the error value when the robotic device moves in the one of the plurality of areas along the movement path.

[0015] The at least one processor of the robotic device may be further configured to execute the at least one instruction to: based on identifying an area among the plurality of areas having a reliability value less than the critical value, modify the movement path to bypass the identified area.

[0016] According to one aspect of the present disclosure, a method for controlling a robot device includes: obtaining a map of a space in which the robot device is located, wherein the map includes multiple areas and each of the multiple areas has a corresponding reliability value; identifying at least one area of ​​the multiple areas having a reliability value greater than or equal to a critical value based on the reliability value of each area of ​​the multiple areas; and identifying a movement path of the robot device in the space based on the at least one area.

[0017] The method for controlling the robot device may also include: estimating one of the multiple areas as the position of the robot device based on detection data of the sensor of the robot device; and obtaining an estimated reliability value corresponding to the estimated position of the robot device by determining the probability that the estimated position of the robot device and the actual position of the robot device match each other.

[0018] The estimating step may include: acquiring movement information of the robot device based on first detection data and second detection data continuously received from the sensor; and estimating a new position of the robot device corresponding to the second detection data based on the movement information.

[0019] The sensor may include a light detection and ranging (LiDAR) sensor, the first detection data may include first point cloud data received from the LiDAR sensor, the second detection data may include second point cloud data received from the LiDAR sensor, and obtaining the estimated reliability value may include: performing an operation of combining a plurality of point clouds based on the first point cloud data and the second point cloud data; and based on a failure of the operation of combining the plurality of point clouds, obtaining an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value.

[0020] The sensor may include a camera, the first detection data may include first image data received from the camera, the second detection data may include second image data received from the camera, and obtaining the estimated reliability value may include: identifying dynamic objects based on the first image data and the second image data; and based on the number of dynamic objects being greater than or equal to a critical number, obtaining an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value.

[0021] According to one aspect of the present disclosure, a non-transitory computer-readable medium includes instructions stored therein, wherein the instructions, when executed by a processor, cause the processor to perform a method of controlling a robotic device, the method comprising: obtaining a map of a space in which the robotic device is located, wherein the map includes multiple areas, and each of the multiple areas has a corresponding reliability value; identifying at least one area of ​​the multiple areas having a reliability value greater than or equal to a critical value based on the reliability value of each area in the multiple areas; and identifying a movement path of the robotic device in the space based on the at least one area.

[0022] The non-temporary computer-readable medium, wherein the method may also include: estimating one of the multiple areas as the position of the robot device based on detection data of the sensor of the robot device; and obtaining an estimated reliability value corresponding to the estimated position of the robot device by calculating the probability that the estimated position of the robot device and the actual position of the robot device match each other.

[0023] The non-transitory computer-readable medium, wherein the estimating step may include: acquiring movement information of the robot device based on first detection data and second detection data continuously received from the sensor; and estimating a new position of the robot device corresponding to the second detection data based on the movement information.

[0024] The non-transitory computer-readable medium, wherein the sensor may include a light detection and ranging (LiDAR) sensor, the first detection data may include first point cloud data received from the LiDAR sensor, the second detection data may include second point cloud data received from the LiDAR sensor, and obtaining an estimated reliability value may include: attempting to combine a plurality of point clouds based on the first point cloud data and the second point cloud data; and based on a failure of attempting to combine the plurality of point clouds, obtaining an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value.

[0025] The non-transitory computer-readable medium, wherein the sensor may include a camera, the first detection data may include first image data received from the camera, the second detection data may include second image data received from the camera, and obtaining the estimated reliability value may include: identifying dynamic objects based on the first image data and the second image data; and based on the number of dynamic objects being greater than or equal to a critical number, obtaining an updated reliability value corresponding to the new position of the robotic device, wherein the updated reliability value is less than the critical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which: Figure 1 is a view for explaining a robot apparatus according to one or more embodiments of the present disclosure; Figure 2 is a block diagram showing a configuration of a robot apparatus according to one or more embodiments of the present disclosure; Figure 3 is a view for explaining a map of a space in which a robot apparatus according to one or more embodiments of the present disclosure is located; Figure 4 is a diagram for explaining movement information of a robot apparatus according to one or more embodiments of the present disclosure; Figure 5 is a view for explaining a plurality of areas included in a map according to one or more embodiments of the present disclosure; Figure 6is a view for explaining a method of estimating a position of a robot device according to one or more embodiments of the present disclosure; Figure 7 is a view for explaining point cloud data according to one or more embodiments of the present disclosure; Figure 8 is a view for explaining a dynamic object according to one or more embodiments of the present disclosure; Fig. 9 is a view for explaining a field of view (FOV) according to one or more embodiments of the present disclosure; and Fig.10 is a flowchart for explaining a control method of a robot apparatus according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0028] Considering the functions in the present disclosure, currently widely used general terms are selected as the terms used in the embodiments of the present disclosure, and may be changed based on the intention of technicians in the field or the emergence of judicial precedents, new technologies, etc. In addition, in certain cases, there may be terms arbitrarily selected by the applicant. In this case, the meanings of these terms are mentioned in detail in the corresponding description of the present disclosure. Therefore, the terms used in the embodiments of the present disclosure need to be defined on the basis of the meaning of the terms and the content throughout the present disclosure, rather than on the basis of the simple names of the terms.

[0029] In the present disclosure, expressions “having”, “may have”, “including”, “may include”, etc. indicate the existence of corresponding features (eg, values, functions, operations, or components such as parts), and do not exclude the existence of additional features.

[0030] The expression “at least one of A or B” may indicate only A, only B, or both A and B.

[0031] The expressions "first", "second", etc. used in the present disclosure may indicate various components regardless of the order or importance of the components. These expressions are only used to distinguish one component from another component and do not limit the corresponding components.

[0032] In the case where it is mentioned that any component (e.g., a first component) is "(operably or communicatively) coupled" to another component (e.g., a second component) / "(operably or communicatively) coupled to" another component (e.g., a second component) or "connected to" another component (e.g., a second component), it should be understood that any component may be directly coupled to another component or may be coupled to another component through yet another component (e.g., a third component).

[0033] Unless the context clearly indicates otherwise, a singular term may include its plural form. It should be understood that the terms "comprising", "consisting of", etc. used in this application specify the presence of the features, numbers, steps, operations, components, parts or combinations thereof mentioned in the specification, and do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0034] In the embodiments, a "module" or "device" may perform at least one function or operation and may be implemented by hardware or software, or by a combination of hardware and software. In addition, in addition to a "module" or "device" that needs to be implemented by specific hardware, multiple "modules" or multiple "devices" may be integrated into at least one module and may be implemented by at least one processor.

[0035] In the specification, a term such as “user” may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) that uses the electronic device.

[0036] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0037] Figure 1 is a view for explaining a robot apparatus according to one or more embodiments of the present disclosure.

[0038] like Figure 1 As shown, the robot device 100 may refer to various types of devices having the ability to perform functions for itself. For example, the robot device 100 may be an intelligent device that detects the surrounding environment of the robot device 100 in real time based on detection data of a sensor (e.g., a light detection and ranging (LiDAR) sensor or a camera) and collects information in addition to performing simple and repetitive functions.

[0039] The robot device 100 according to one or more embodiments of the present disclosure may include a driver including an actuator or a motor. The driver according to one or more embodiments may include wheels, brakes, etc., and the robot device 100 may move itself in a specific space by using the wheels, brakes, etc. included in the driver.

[0040] The robot apparatus 100 according to one or more embodiments may control the movement (or connection) of the robot joint by using a driver. Here, the robot joint may be a component of the robot apparatus 100 for replacing the function of a human arm or hand.

[0041] The robot device 100 according to one or more embodiments of the present disclosure may detect the surrounding environment of the robot device 100 in real time based on the detection data of the sensor. Next, the robot device 100 may control the actuator based on the detected surrounding environment. For example, the robot device 100 may identify the movement path of the robot device 100 based on the detection data of the sensor.

[0042] For example, the robot device 100 may acquire a map of the space where the robot device 100 is located based on the detection data, and identify the position of the robot device 100 on the map. For example, the robot device 100 may identify its position in real time and move along a movement path in the space. Therefore, the robot device 100 may need to accurately (or with high reliability) identify its position in order to move along the movement path in the space.

[0043] Hereinafter, the specification describes various embodiments in which the robot device 100 recognizes its position on a map and recognizes a movement path by considering whether the recognized position matches the current position (or actual position) of the robot device 100 in space.

[0044] The robot device 100 may be classified into a robot device for industrial, medical, domestic, military, or exploration use based on its field or executable functions. According to one or more embodiments, the industrial robot device may be subdivided into a robot device used in a product manufacturing process in a factory, a robot device serving customers, receiving orders, and providing services in a store or restaurant, etc. For example, Figure 1 As shown, the robot device 100 can be implemented as a service robot device, which can deliver service items to a user's desired location or to a specific location in any of various places such as restaurants, hotels, shopping malls, hospitals, clothing stores, etc.

[0045] However, such implementation is merely an example, and the robot device 100 may be classified into various types based on its application field, function, and purpose of use, and is not limited to the above example.

[0046] For example, Figure 6 As shown, the robot device 100 may be implemented as a robot cleaner located in a house. Here, the robot cleaner may be a device that is driven by electricity and automatically sucks foreign matter. Hereinafter, for ease of explanation, it is assumed that the robot device 100 is a robot cleaner, and the robot cleaner is implemented as a flat type that is in close contact with the floor to suck foreign matter on the floor. However, this implementation is only an example, and the robot device 100 may be implemented in various forms as described above.

[0047] Figure 2is a block diagram illustrating a configuration of a robot apparatus according to one or more embodiments of the present disclosure.

[0048] The robot device 100 according to one or more embodiments may include a memory 110 , a sensor 120 , and at least one processor 130 (hereinafter referred to as a processor).

[0049] The memory 110 according to one or more embodiments may store data required by various embodiments of the present disclosure. Based on the data storage purpose, the memory 110 may be implemented as a memory embedded in the robot device 100, or as a memory detachable from the robot device 100. For example, data for driving the robot device 100 may be stored in a memory embedded in the robot device 100, and data for extended functions of the robot device 100 may be stored in a memory detachable from the robot device 100.

[0050] The memory embedded in the robot device 100 may be implemented as at least one of a volatile memory (e.g., a dynamic random access memory (DRAM), a static RAM (SRAM), or a synchronous dynamic RAM (SDRAM)), a nonvolatile memory (e.g., a one-time programmable read-only memory (OPTROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a mask ROM or a flash ROM, a flash memory (e.g., a NAND flash memory or a NOR flash memory), a hard disk drive, or a solid state drive (SSD)). In addition, the memory removable from the robot device 100 may be implemented in the form of a memory card (e.g., a compact flash (CF), a secure digital (SD), a micro secure digital (Micro-SD), a mini secure digital (Mini-SD), an extreme digital (xD), or a multimedia card (MMC)) or an external memory (e.g., a USB memory) connectable to a universal serial bus (USB) port.

[0051] For example, the memory 110 may store at least one instruction for controlling the robotic device 100 or a computer program including the instruction.

[0052] According to one or more embodiments of the present disclosure, various data may be stored in an external memory of the processor 130 , some of the data may be stored in an internal memory of the processor 130 , and the rest of the data may be stored in the external memory.

[0053] Specifically, the memory 110 may store a map of a space where the robot device 100 is located under the control of the processor 130 .

[0054] For example, the processor 130 may acquire a map of a space where the robot 100 is located based on the detection data received from the sensor 120 , and store the map in the memory 110 .

[0055] Here, the sensor 120 may include a LiDAR sensor, a camera, or the like.

[0056] For example, the processor 130 may detect the surrounding environment of the robot device 100 by controlling the LiDAR sensor to emit a laser beam, and the LiDAR sensor may acquire the distance to an object adjacent to the robot device 100, the direction in which the object is located relative to the robot device 100, and the characteristics of the object as detection data. Next, the processor 130 may acquire the surrounding environment of the robot device 100 as two-dimensional (2D) / three-dimensional (3D) image information (e.g., a map) based on the detection data.

[0057] For example, the processor 130 may acquire image data by controlling the camera to detect the surroundings of the robot device 100. Here, the camera may acquire image data by capturing the surroundings of the robot device 100 under the control of the processor 130, and transmit the acquired image data to the processor 130. Next, the processor 130 may analyze the image data to acquire the distance to an object adjacent to the robot device 100, the direction in which the object is located relative to the robot device 100, and the features of the object, and acquire the surroundings of the robot device 100 as 2D / 3D image information (e.g., a map).

[0058] The sensor 120 is not limited to the above-mentioned examples, and may include various types of sensors that can detect the surrounding environment of the robot device 100 in addition to the LiDAR sensor or the camera.

[0059] At least one processor 130 according to one or more embodiments of the present disclosure may control overall operations of the robot device 100 .

[0060] According to one or more embodiments of the present disclosure, the processor 130 may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) for processing digital signals. However, the processor 130 is not limited thereto, and may include one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessor unit (MPU), a controller, an application processor (AP), a communication processor (CP), an advanced reduced instruction set computer (RISC) machine (ARM) processor, or an artificial intelligence (AI) processor, or may be defined by the term. In addition, the processor 130 may be implemented in a system on chip (SoC) or a large-scale integrated circuit (LSI) embedded with a processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA). The processor 130 may perform various functions by executing computer executable instructions stored in a memory.

[0061] Figure 3 is a view for explaining a map of a space in which a robot apparatus is located according to one or more embodiments of the present disclosure.

[0062] The processor 130 according to one or more embodiments of the present disclosure may acquire a map of a space in which the robot device 100 is located by performing a simultaneous localization and mapping (SLAM) operation.

[0063] Here, the SLAM operation may be an operation of acquiring a map of a space where the robot device 100 is located using detection data received from the sensor 120 and recognizing a position of the robot device 100 on the map.

[0064] For example, the processor 130 may estimate the position of the robotic device 100 on the map based on the detection data received from the sensor 120 .

[0065] For example, in a case of sequentially (or continuously) receiving the first detection data and the second detection data, the processor 130 may acquire movement information of the robot device 100 based on the first detection data and the second detection data.

[0066] Next, the processor 130 may estimate the position of the robot device 100 on the map at the time point when the second detection data is received based on the movement information.

[0067] Figure 4 is a view for explaining movement information of a robot apparatus according to one or more embodiments of the present disclosure.

[0068] refer to Figure 4 , the sensor 120 in the robot device 100 can obtain the first detection data at the time point t, and send the obtained first detection data to the processor 130.

[0069] Next, the sensor 120 located in the robot device 100 ′ (the robot device 100 ′ is the same as the robot device 100 but shifted forward in time) may acquire second detection data at the time point t+1 and send the acquired second detection data to the processor 130 .

[0070] Next, the processor 130 may acquire movement information of the robot device 100 from time point t to time point t+1 based on the first detection data and the second detection data. The processor 130 according to one or more embodiments may estimate the position of the robot device 100' on the map at time point t+1 based on the movement information.

[0071] For example, the sensor 120 may include a LiDAR sensor. The processor 130 may combine (register or align) the point clouds with each other based on the first point cloud data included in the first detection data received from the LiDAR sensor and the second point cloud data included in the second detection data received from the LiDAR sensor.

[0072] For example, the processor 130 may combine the first point cloud data and the second point cloud data with each other by using a LiDAR odometry method such as an iterative closest point (ICP) algorithm or a normal distribution transform (DNT). Next, the processor 130 may estimate the position (or movement trajectory) of the robot device 100' based on the result of combining the first point cloud data and the second point cloud data with each other.

[0073] For example, the sensor 120 may include a camera. The processor 130 may acquire movement information of the robot device 100 from time point t to time point t+1 based on the first image data included in the first detection data received from the camera and the second image data included in the second detection data received from the camera.

[0074] For example, the processor 130 may analyze the first image data to identify features of at least one object included in the first image data (eg, geometric features of at least one object), and analyze the second image data to identify features of at least one object included in the second image data.

[0075] Next, the processor 130 may acquire movement information of the robot device 100 from the time point t to the time point t+1 by comparing features of at least one object included in the first image data and features of at least one object included in the second image data with each other.

[0076] Next, the processor 130 may estimate the position (or movement trajectory) of the robot device 100 ′ based on the movement information.

[0077] Here, the sensor 120 may include at least one of a LiDAR sensor or a camera.

[0078] The processor 130 according to one or more embodiments may estimate a real-time position of the robot device 100 on the map based on a plurality of detection data received in real time or sequentially from the sensor 120 while the robot device 100 moves in the space.

[0079] The processor 130 according to one or more embodiments may identify a probability that an estimated position of the robot device 100 on the map and an actual position thereof in the space match each other.

[0080] For example, the processor 130 may identify whether the estimated position of the robot device 100 is reliable as its actual position, that is, the probability (hereinafter, reliability value) that the estimated position matches its actual position.

[0081] Here, the reliability value may represent the reliability of the estimated position based on the movement information, and thus is inversely proportional to the position measurement failure value (or positioning failure value (LFV)).

[0082] For example, in the case where the reliability value is high, the estimated position of the robot device 100 based on the movement information is likely to match its actual position, and the positioning failure value may be low. On the other hand, in the case where the reliability value is low, the estimated position of the robot device 100 based on the movement information is almost unlikely to match its actual position, and the positioning failure value may be high.

[0083] The processor 130 according to one or more embodiments may classify the map into a plurality of regions, and acquire and store a reliability value for each of the plurality of regions.

[0084] Figure 5 is a view for explaining a plurality of areas included in a map according to one or more embodiments of the present disclosure.

[0085] refer to Figure 5 , a map may be classified into a plurality of regions. Each of the plurality of regions may be referred to as a unit, and for the convenience of explanation, they are collectively referred to as regions hereinafter. In addition, Figure 5 Each size in the plurality of regions shown in is an example and is not limited thereto.

[0086] The processor 130 according to one or more embodiments may estimate a real-time position of the robot device 100 based on a plurality of detection data received in real time or sequentially from the sensor 120 while the robot device 100 moves.

[0087] Next, the processor 130 may identify a reliability value (ie, a probability that the estimated position of the robot device 100 matches its actual position), and map the identified reliability value and a region corresponding to the estimated position among the plurality of regions to each other.

[0088] Therefore, the processor 130 may identify a reliability value of each of the plurality of regions and acquire a reliability value map (or a positioning failure value (LFV)) map (hereinafter referred to as a LFV map) corresponding to the map.

[0089] For example, in a case where the estimated position corresponds to any one of a plurality of regions of the robot device 100, the processor 130 may acquire a probability that the estimated position of the robot device 100 matches its actual position from the reliability value map.

[0090] The processor 130 may need to identify a suitable movement path in the space and may accurately (or with high reliability) estimate the position of the robotic device on the map to accurately move the robotic device 100 along the identified movement path.

[0091] The processor 130 according to one or more embodiments may identify at least one region having a reliability value greater than or equal to a critical value based on the reliability value of each of the plurality of regions included in the reliability value map. Next, the processor 130 may identify a movement path of the robot device 100 in space based on the identified at least one region.

[0092] For example, the processor 130 may identify, among a plurality of movement paths having the same start point and end point, a movement path in which the robot device is preferentially moved in an area having a reliability value greater than or equal to a critical value based on the reliability value map.

[0093] For example, the processor 130 may identify, among a plurality of moving paths having the same start point and end point, a moving path that bypasses an area having a reliability value less than a critical value based on the reliability value map.

[0094] Next, the processor 130 may move the robotic device 100 based on the identified movement path.

[0095] Hereinafter, the specification describes various embodiments in which the processor 130 identifies a reliability value.

[0096] Figure 6 is a view for explaining a method of estimating a position of a robot device according to one or more embodiments of the present disclosure.

[0097] refer to Figure 6, the processor 130 may combine (register or align) the point clouds with each other based on the first point cloud data included in the first detection data received from the LiDAR sensor and the second point cloud data included in the second detection data received from the LiDAR sensor.

[0098] like Figure 6 As shown, the processor 130 may combine point clouds with each other based on first point cloud data and second point cloud data corresponding to an object (eg, a sofa) adjacent to the robot device 100 .

[0099] Next, the processor 130 may acquire movement information based on the combined result, and estimate the position of the robot device 100 on the map based on the acquired movement information.

[0100] In addition, the processor 130 may acquire movement information of the robot device 100 from time point t to time point t+1 by identifying commonality / similarity between geometric features of an object acquired from each of first image data included in the first detection data received from the camera and second image data included in the second detection data. In addition, the processor 130 may estimate the position of the robot device based on the movement information.

[0101] For example, the processor 130 may identify a reliability value of a region corresponding to the estimated position among the plurality of regions as being greater than or equal to a critical value.

[0102] In some cases, no object may exist near the robot device 100 (or a few objects may exist near the robot device 100). In this case, the processor 130 cannot obtain the first point cloud data and the second point cloud data, and thus may not be able to combine the point clouds with each other. In this case, the processor 130 cannot obtain movement information based on the combination result, and may not be able to estimate the position of the robot device 100 on the map. Figure 7 Provide a detailed description thereof.

[0103] Figure 7 is a view for explaining point cloud data according to one or more embodiments of the present disclosure.

[0104] like Figure 7 As shown, in the case where there are a few objects near the robot device 100, the processor 130 may have a small number of points included in the point cloud data. Therefore, in the case where the number of points included in the point cloud data is small, the processor 130 may recognize that the reliability value of the area corresponding to the estimated position is less than the critical value.

[0105] The processor 130 according to one or more embodiments may acquire movement information by combining the point cloud of the first point cloud data and the point cloud of the second point cloud data with each other, and estimate the position of the robot device 100 based on the acquired movement information. In the case where the number of points included in each of the first point cloud data and the second point cloud data is small, the processor 130 may recognize that the reliability value of the area corresponding to the estimated position is less than a critical value.

[0106] Furthermore, in the case where each of the first point cloud data and the second point cloud data does not include any point, the processor 130 may be unable to acquire movement information and estimate the position of the robot device 100 on the map.

[0107] The processor 130 may identify, among a plurality of movement paths having the same start point and end point, a movement path in which the robot device is preferentially moved through an area having a reliability value greater than or equal to a critical value based on the reliability value map.

[0108] For example, refer to Figure 6 The moving path indicated by the dashed line in FIG. 1 may include an area having a reliability value less than a critical value. Therefore, the processor 130 may identify the area including Figure 6 The moving path indicated by the solid line (ie, the region having a reliability value greater than or equal to the critical value) is the moving path.

[0109] Figure 8 is a view for explaining a dynamic object according to one or more embodiments of the present disclosure.

[0110] refer to Figure 8 , the object adjacent to the robot device 100 may be a dynamic object. In this case, the processor 130 may estimate the position of the robot device based on the point cloud data corresponding to the dynamic object, and identify that the reliability value of the area corresponding to the estimated position is less than a critical value.

[0111] For example, the processor 130 may acquire movement information by combining the point cloud of the first point cloud data and the point cloud of the second point cloud data with each other, and estimate the position of the robot device 100 based on the acquired movement information. In the case where each of the first point cloud data and the second point cloud data includes a point cloud corresponding to a dynamic object, the processor 130 may recognize that the reliability value of the area corresponding to the estimated position is less than a critical value.

[0112] For example, a dynamic object may not move like a static object, and the movement information of the robot apparatus 100 from the time point t to the time point t+1 may include the movement of the dynamic object in addition to the movement of the robot apparatus 100 .

[0113] For example, in the case of estimating the position of the robot device 100 based on the movement information, the processor 130 may consider the movement of the dynamic object in addition to the movement of the robot device 100 to estimate the position of the robot device 100. Therefore, an error may occur between the estimated position of the robot device 100 and its actual position. Therefore, in the case where each of the first point cloud data and the second point cloud data includes a point cloud corresponding to the dynamic object, the processor 130 may recognize that the reliability value of the area corresponding to the estimated position is less than the critical value.

[0114] The sensor 120 may include a camera, and the processor 130 may identify a dynamic object based on first image data included in the first detection data received from the camera and second image data included in the second detection data received from the camera, and may identify that a reliability value of an area corresponding to the estimated position is less than a critical value when a dynamic object is identified (or the number of identified dynamic objects is greater than or equal to a critical number).

[0115] Fig. 9 is a view for explaining a field of view (FOV) according to one or more embodiments of the present disclosure.

[0116] refer to Fig. 9 , the processor 130 may identify the field of view based on the detection data received from the sensor.

[0117] In the case where the identified field of view is smaller than the critical value, the processor 130 may identify that the reliability value of the area corresponding to the estimated position is smaller than the critical value.

[0118] like Fig. 9 As shown, the processor 130 may identify an object (eg, obstacle 10 ) located near a left direction or a right direction based on a moving direction (eg, a traveling direction) of the robot device 100 based on image data received from the camera.

[0119] Next, the processor 130 may identify the field of view of the robot device 100 based on the identified object. In the case of identifying that the angle of the field of view is less than a critical angle, the processor 130 according to one or more embodiments may identify that a reliability value of a region corresponding to a position estimated from the received image data is less than a critical value.

[0120] For example, when identifying an object located near the left or right direction based on the moving direction of the robot device 100 or the robot device 100 moves along a narrow path, it may be difficult for the processor 130 to recognize the geometric features of the object, the photometric features of the object, etc. from the image data received from the camera.

[0121] For example, in the case where the geometric features of the object cannot be recognized (or it is difficult) from each of the first image data included in the first detection data and the second image data included in the second detection data, the processor 130 may not be able to acquire the movement information of the robot device 100 from the time point t to the time point t+1. Therefore, the processor 130 may not be able to estimate the position of the robot device 100 on the map.

[0122] Furthermore, in the case where the processor 130 fails to recognize (or has difficulty in recognizing) commonality / similarity between geometric features of an object acquired from each of the first image data included in the first detection data and the second image data included in the second detection data, the probability that the position of the robot device 100 estimated from the movement information of the robot device 100 from the time point t to the time point t+1 matches its actual position may be low. Therefore, the processor 130 may recognize that the reliability value of the area corresponding to the estimated position is less than the critical value.

[0123] return Figure 2 In addition to the detection data received from the sensor 120, the processor 130 according to one or more embodiments of the present disclosure may also acquire movement information based on the number of rotations of a wheel included in the driver, a rotation speed of the wheel, and the like.

[0124] The processor 130 according to one or more embodiments may obtain movement information of the robot device 100 from time point t to time point t+1, and upon identifying that at least one of the movement direction, movement distance, or movement speed of the robot device 100 included in the identified movement information cannot be performed by a driver located in the robot device 100, identify that the reliability value of the area corresponding to the estimated position is less than a critical value.

[0125] For example, in a case where it is identified that a driver located in the robot device 100 can be moved only forward or backward, and the robot device 100 is moved left or right based on the moving direction of the robot device 100 included in the identified movement information, the processor 130 may identify that the reliability value of the area corresponding to the estimated position is less than a critical value.

[0126] For example, when it is identified that the maximum speed of the driver located in the robot device 100 is 5 m / s, and the robot device is moved at 10 m / s based on the moving distance and moving speed of the robot device 100 included in the identified movement information, the processor 130 can identify that the reliability value of the area corresponding to the estimated position is less than the critical value.

[0127] According to one or more embodiments, the processor 130 may identify a moving path based on a map and a reliability value graph. In detail, the processor 130 may identify a moving path including at least one area having a reliability value greater than or equal to a critical value based on the reliability value of each area in a plurality of areas included in the map.

[0128] In detail, the processor 130 may identify a moving path that bypasses at least one area having a reliability value less than a critical value based on the reliability value of each of a plurality of areas included in the map.

[0129] The processor 130 may acquire reliability values ​​while the robot device 100 moves along the movement path, and update the reliability value map based on the acquired reliability values.

[0130] The sensor 120 according to one or more embodiments may include a LiDAR sensor, and the processor 130 may identify an error value based on the orientation of the LiDAR sensor in any one of a plurality of areas based on detection data received from the LiDAR sensor.

[0131] Next, while the robot device 100 moves in an area along the moving path, the processor 130 may adjust the posture of the LiDAR sensor based on the error value.

[0132] For example, a covariance value or an error value of detection data of a LiDAR sensor may depend on a direction in which a laser beam is emitted by the LiDAR sensor at the same location.

[0133] According to one or more embodiments, the processor 130 may identify an orientation of a LiDAR sensor having a low covariance value and a low error value if the robot device moves in a specific area, based on detection data received from the LiDAR sensor.

[0134] Next, the processor 130 may adjust the orientation of the LiDAR sensor to emit a laser beam so as to have a low covariance value and a low error value when the robot device 100 moves in a specific area along a moving path, instead of matching the moving direction of the robot device 100 with the orientation of the LiDAR sensor (i.e., the emission direction of the laser beam).

[0135] Therefore, the moving direction of the robot device 100 and the direction in which the LiDAR sensor emits the laser beam may be the same as or different from each other.

[0136] According to various embodiments of the present disclosure, as the covariance value or the error value is reduced based on the detection data of the LiDAR sensor, the probability that the estimated position and the actual position match each other may increase, and as the number of points included in the point cloud data increases, the probability that the estimated position and the actual position match each other may increase. In addition, in the case where the object adjacent to the robot device 100 is not a dynamic object but a static object, the probability that the estimated position and the actual position match each other may increase.

[0137] According to various embodiments of the present disclosure, as the covariance value or the error value is reduced based on the detection data of the camera, the probability that the estimated position and the actual position match each other can be increased, and in the case where the geometric features of the object included in the image data or the photometric features of the object are recognized, the probability that the estimated position and the actual position match each other can be increased. In addition, in the case where the object adjacent to the robot device 100 is not a dynamic object but a static object, the probability that the estimated position and the actual position match each other can be increased.

[0138] The processor 130 according to one or more embodiments of the present disclosure may identify a movement path based on the reliability value map, thereby moving the robot device 100 to be adjacent to a static object (eg, a wall, furniture, or home appliance) in a space and to bypass a dynamic object.

[0139] In addition, the processor 130 may identify a moving path based on the reliability value map, thereby moving the robot device 100 to bypass a narrow path in the space, or allowing the detection direction of the sensor 120 (e.g., the posture of the LiDAR sensor) to not match the moving direction of the robot device 100 when the robot device moves in a narrow path.

[0140] The robot device 100 according to one or more embodiments of the present disclosure may include a communication interface. The communication interface may receive various data. For example, the communication interface may receive various data from at least one external device located in the house, an external storage medium (e.g., a universal serial bus (USB) memory), an external server (e.g., a network hardware or a streaming media server), etc. by using a communication method such as wireless fidelity (Wi-Fi, i.e., wireless local area network (LAN)) based on an access point (AP), Bluetooth, Zigbee, a wired / wireless local area network (LAN), a wide area network (WAN), Ethernet, IEEE 1394, a high-definition multimedia interface (HDMI), USB, a mobile high-definition link (MHL), an audio engineering society / European broadcasting union (AES / EBU) communication, optical communication, or coaxial communication.

[0141] Fig.10 is a flowchart for explaining a control method of a robot apparatus according to one or more embodiments of the present disclosure.

[0142] According to one or more embodiments of the present disclosure, a control method for a robot device including a map of a space in which the robot device is located and a reliability value of each of a plurality of regions in the map may first include identifying at least one region having a reliability value greater than or equal to a critical value based on the reliability value of each of the plurality of regions (S1010).

[0143] Subsequently, the method may include identifying a movement path of the robot device in the space based on the identified at least one region ( S1020 ).

[0144] The control method according to one or more embodiments may further include: estimating any one of a plurality of areas as the position of the robot device based on detection data of a sensor included in the robot device; and obtaining a probability that the estimated position of the robot device and its actual position match each other as a reliability value.

[0145] Here, the estimating step may include: acquiring movement information of the robot apparatus based on the first detection data and the second detection data when first detection data and second detection data are continuously received from the sensor; and estimating a position of the robot apparatus corresponding to the second detection data based on the movement information.

[0146] Here, obtaining the probability may include: combining the point clouds with each other based on first point cloud data included in the first detection data received from the LiDAR sensor and second point cloud data included in the second detection data received from the LiDAR sensor; and in the event of a combination failure, obtaining a reliability value less than a critical value.

[0147] Obtaining the probability according to one or more embodiments may include: identifying dynamic objects based on first image data included in first detection data received from the camera and second image data included in second detection data received from the camera; and obtaining a reliability value less than the critical value when the number of identified dynamic objects is greater than or equal to a critical value.

[0148] Obtaining the probability according to one or more embodiments may include: identifying a field of view of a camera based on image data received from the camera and included in detection data; and obtaining a reliability value less than a critical value when an angle of the identified field of view is less than a critical angle.

[0149] Acquiring the probability according to one or more embodiments may include acquiring a reliability value less than a critical value when at least one of a moving direction, a moving distance, or a moving speed of the robot apparatus included in the movement information is identified as being impossible to be performed by the robot apparatus.

[0150] The control method according to one or more embodiments may further include: updating a reliability value of any one of the plurality of regions based on the acquired reliability value.

[0151] The control method according to one or more embodiments may further include: identifying an error value based on a posture of the LiDAR sensor in one of the plurality of areas based on detection data received from the LiDAR sensor; and adjusting the posture of the LiDAR sensor based on the error value when the robot device moves in one area along a moving path.

[0152] Operation S1020 of identifying the moving path according to one or more embodiments may include, in a case where a region having a reliability value less than a critical value is identified based on the reliability value of each of the plurality of regions, identifying a moving path that bypasses the identified region.

[0153] However, various embodiments of the present disclosure may be applied to all movable electronic devices as well as robot devices.

[0154] The various embodiments described above can be implemented in a computer or computer-readable medium recording using software, hardware, or a combination of software and hardware. In some cases, the embodiments described in the specification can be implemented by the processor itself. According to software implementation, embodiments such as the processes and functions described in the specification can be implemented by separate software modules. Each software module can perform one or more functions and operations described in the specification.

[0155] Computer instructions for executing the processing operations of the robot device according to the various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. In the case where the computer instructions are executed by a processor of a specific device, the computer instructions stored in the non-transitory computer-readable medium may allow the specific device to execute the processing operations of the robot device 100 according to the various embodiments described above.

[0156] A non-transitory computer-readable medium is not a medium in which data is temporarily stored, such as a register, cache, or memory, but refers to a medium in which data is semi-permanently stored and can be read by a machine. Specific examples of non-transitory computer-readable media may include: compact disc (CD), digital versatile disc (DVD), hard disk, Blu-ray disc, universal serial bus (USB), memory card, read-only memory (ROM), etc.

[0157] Although the embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the above specific embodiments, and various modifications may be made by those skilled in the art to which the present disclosure belongs without departing from the scope and spirit of the present disclosure disclosed in the appended claims. These modifications should also be understood to fall within the scope and spirit of the present disclosure.

Claims

1. A robotic device comprising: at least one memory storing at least one instruction; a sensor configured to detect an environment of the robotic device and output detection data; as well as at least one processor configured to execute the at least one instruction to: acquiring a map of a space in which the robot device is located and a reliability value of each of a plurality of areas in the map based on the detection data received from the sensor, storing the map and the reliability value of each of the plurality of regions in the at least one memory, identifying at least one region having a reliability value greater than or equal to a critical value based on the reliability value of each region of the plurality of regions, and A movement path of the robotic device in the space is identified based on the at least one area.

2. The robot device according to claim 1, wherein: The at least one processor may be further configured to execute the at least one instruction to: estimating one of the plurality of areas as the position of the robotic device based on the detection data, and An estimated reliability value corresponding to the estimated position of the robot apparatus is obtained by determining a probability that the estimated position of the robot apparatus and the actual position of the robot apparatus match each other.

3. The robot device according to claim 2, wherein: The at least one processor is further configured to execute the at least one instruction to: acquiring movement information of the robot device based on first detection data and second detection data continuously received from the sensor, and A new position of the robot apparatus corresponding to the second detection data is estimated based on the movement information.

4. The robot device according to claim 3, in, The sensors include light detection and ranging (LiDAR) sensors, wherein the first detection data includes first point cloud data received from the LiDAR sensor, and the second detection data includes second point cloud data received from the LiDAR sensor, and Wherein, the at least one processor is further configured to execute the at least one instruction to: performing an operation of combining a plurality of point clouds based on the first point cloud data and the second point cloud data, and Based on a failure of combining the plurality of point clouds, an updated reliability value corresponding to the new position of the robotic device is obtained, wherein the updated reliability value is less than the critical value.

5. The robot device according to claim 3, wherein: The sensor includes a camera, wherein the first detection data comprises first image data received from the camera, and the second detection data comprises second image data received from the camera, and Wherein, the at least one processor is further configured to execute the at least one instruction to: identifying a dynamic object based on the first image data and the second image data, and Based on the number of dynamic objects being greater than or equal to a critical number, an updated reliability value corresponding to the new position of the robotic device is obtained, wherein the updated reliability value is less than the critical value.

6. The robot device according to claim 3, wherein: The sensor includes a camera, wherein the detection data includes image data received from the camera, and Wherein, the at least one processor is further configured to execute the at least one instruction to: identifying a field of view of the camera based on the image data, and Based on the angle of the field of view being less than a critical angle, an updated reliability value corresponding to the new position of the robotic device is obtained, wherein the updated reliability value is less than the critical value.

7. The robot device according to claim 3, wherein: The movement information includes: the movement direction of the robot device, the movement distance of the robot device, or the movement speed of the robot device, and Wherein, the at least one processor is further configured to execute the at least one instruction to: Based on identifying that at least one of the moving direction of the robot device, the moving distance of the robot device and the moving speed of the robot device cannot be maintained, an updated reliability value corresponding to the new position of the robot device is obtained, wherein the updated reliability value is less than the critical value.

8. The robot device according to claim 2, wherein: The at least one processor is further configured to execute the at least one instruction to update a reliability value corresponding to one of the plurality of regions into the at least one memory based on the obtained reliability value.

9. The robot device according to claim 2, wherein: The sensors include light detection and ranging (LiDAR) sensors, and Wherein, the at least one processor is further configured to execute the at least one instruction to: identifying, based on the detection data received from the LiDAR sensor, an error value based on the orientation of the LiDAR sensor in one of the plurality of regions, and When the robotic device moves in the one of the plurality of areas along the movement path, the orientation of the LiDAR sensor is adjusted based on the error value.

10. The robot device according to claim 1, wherein: The at least one processor is further configured to execute the at least one instruction to: Based on identifying an area among the plurality of areas having a reliability value less than the critical value, the movement path is modified to bypass the identified area.

11. A method of controlling a robotic device, the method comprising: Acquire a map of a space where the robot device is located, wherein the map includes a plurality of regions, and each of the plurality of regions has a corresponding reliability value; identifying, based on the reliability value of each region in the plurality of regions, at least one region in the plurality of regions having a reliability value greater than or equal to a critical value; and A movement path of the robotic device in the space is identified based on the at least one area.

12. The method of claim 11, further comprising: estimating one of the plurality of areas as a position of the robotic device based on detection data of a sensor of the robotic device; as well as An estimated reliability value corresponding to the estimated position of the robot apparatus is obtained by determining a probability that the estimated position of the robot apparatus and the actual position of the robot apparatus match each other.

13. The method of claim 12, wherein: The estimation steps include: acquiring movement information of the robot device based on first detection data and second detection data continuously received from the sensor; and A new position of the robot apparatus corresponding to the second detection data is estimated based on the movement information.

14. The method of claim 13, wherein: The sensors include light detection and ranging (LiDAR) sensors, wherein the first detection data includes first point cloud data received from the LiDAR sensor, and the second detection data includes second point cloud data received from the LiDAR sensor, and Wherein, obtaining the estimated reliability value includes: performing an operation of combining a plurality of point clouds based on the first point cloud data and the second point cloud data; and Based on a failure of combining the plurality of point clouds, an updated reliability value corresponding to the new position of the robotic device is obtained, wherein the updated reliability value is less than the critical value.

15. The method of claim 13, wherein: The sensor includes a camera, wherein the first detection data comprises first image data received from the camera, and the second detection data comprises second image data received from the camera, and Wherein, obtaining the estimated reliability value comprises: identifying a dynamic object based on the first image data and the second image data; and Based on the number of dynamic objects being greater than or equal to a critical number, an updated reliability value corresponding to the new position of the robotic device is obtained, wherein the updated reliability value is less than the critical value.

Images