Robot

Abstract

A robot according to an aspect of the present disclosure comprises: an input interface which receives a command to move to a first destination corresponding to a designated object; and a processor which sets a second destination spaced apart from the first destination, wherein the processor sets the second destination such that the second destination has a greater distance to the object than the first destination, and controls the robot to move to the second destination and then travel close to the object.

Description

robot

[0001] The present disclosure relates to an autonomous robot, and more particularly, to a robot capable of preventing collisions while driving.

[0002] Robots were developed for industrial use and have played a role in factory automation. Recently, the scope of robotics applications has expanded, with developments in medical and aerospace robots, as well as domestic robots for everyday use. Among these robots, those capable of autonomous navigation are called mobile robots.

[0003] As robot use increases, there is a growing demand for robots that can provide diverse information, entertainment, and services beyond the repetitive performance of simple functions.

[0004] Accordingly, various robots are being developed to provide convenience to people by being placed in homes, restaurants, stores, public places, etc.

[0005] Robots are equipped with multiple sensors to avoid obstacles while driving. For example, optical sensors determine the presence and distance of obstacles based on the amount of light reflected back from the obstacle or the time it takes for the light to be received.

[0006] Prior art document 1 (Korean Patent Publication No. 10-2020-0108631) relates to a safety device for mobile robots to prevent accidents, and prevents accidents of mobile robots by combining a primary safety device using multiple RGB-D cameras and a secondary safety device using a bumper.

[0007] Furthermore, prior art document 2 (US Patent No. US9535421) discloses a serving robot. The serving robot disclosed in prior art document 2 includes a touch-based display that provides an interface for user requests and a container for serving items, which is positioned inside the robot. The user places the serving items into the container of the serving robot, and the serving robot begins delivery upon receiving input from the display regarding the transport location of the serving items.

[0008] Depending on the service location and type of service, the robot may be required to come very close to the service recipient.

[0009]

[0010] The purpose of this disclosure is to provide a robot that can safely move to a location where a service recipient can conveniently receive service.

[0011] The purpose of the present disclosure is to provide optimal obstacle recognition and driving technology that takes into account the characteristics of sensors equipped with a robot.

[0012] The purpose of the present disclosure is to provide a robot that can approach a table as close as possible without causing a collision and stop smoothly even when there is an error in position recognition or the table has moved when providing a serving service in a restaurant.

[0013] The purpose of the present disclosure is to provide a robot that can effectively establish a path and driving method depending on the situation.

[0014] In order to achieve the above or other purposes, a robot according to one aspect of the present disclosure can stably respond to changes in driving area conditions by first moving to a virtual destination based on a fixed destination on a map and then moving to a final destination.

[0015] A robot according to one aspect of the present disclosure can safely move to the closest position without colliding with an object by moving to a destination set at a predetermined distance from an object such as a table and then approaching the object.

[0016] A robot according to one aspect of the present disclosure is capable of driving effectively by changing the driving mode according to the driving situation.

[0017] In order to achieve the above or other purposes, a robot according to one aspect of the present disclosure includes an input interface for receiving a command to move to a first destination corresponding to a designated object, and a processor for setting a second destination spaced apart from the first destination, wherein the processor sets the second destination to have a distance longer than the first destination to the object, and controls the robot to move to the second destination and then drive in proximity to the object.

[0018] A robot according to one aspect of the present disclosure may further include a memory in which a map is stored.

[0019] The first destination may be a destination fixedly stored on the map, and the second destination may be a destination newly created during driving.

[0020] The above memory can store coordinate information and direction information of the first destination.

[0021] The coordinates of the second destination are set to be a predetermined distance away from the coordinates of the first destination in the opposite direction of the direction in which the object is located, and the direction of the second destination can be set in the vertical direction of the edge line of the object.

[0022] A robot according to one aspect of the present disclosure may further include a depth camera.

[0023] The second destination may be set to a position and direction in which the depth camera can view the edge of the object.

[0024] The processor can control the vehicle to drive straight in the direction of the object while resetting the second destination and updating the location of the object seen by the depth camera during the proximity driving.

[0025] The second destination can be determined based on the sensing area range of the depth camera.

[0026] A robot according to one aspect of the present disclosure may further include driving wheels including a left wheel and a rear wheel.

[0027] The second destination may be set to a location and direction in which the object can be approached by driving by rotation of the driving wheel.

[0028] A robot according to one aspect of the present disclosure may further include an obstacle detection sensor that detects the presence of an obstacle and the distance to the obstacle.

[0029] The processor can control the obstacle detection sensor to approach the object at the shortest distance without risk of collision based on sensing data.

[0030] The above proximity driving may be driving that approaches the object while moving straight from the second destination toward the object.

[0031] The object specified above may be a table.

[0032] The processor can control the vehicle to drive in the first driving mode to the second destination, and upon arriving at the second destination, to approach the object in the second driving mode.

[0033] The above first driving mode may be a driving mode that uses location recognition based on SLAM (Simultaneous localization and mapping).

[0034] The above second driving mode may be a driving mode that uses position recognition based on odometry information.

[0035] The robot further includes a wheel sensor, and the odometry information may include sensing data of the wheel sensor.

[0036] The robot further includes a lidar sensor, and the odometry information may be based on sensing data of the wheel sensor and the lidar sensor.

[0037] The above processor can reset the position before the proximity driving start to the reference origin coordinates of the odometry information and control the proximity driving to start.

[0038] The processor may rotate the robot so that the robot's direction matches the vertical direction of the edge line of the object, if the robot's direction does not match the vertical direction of the edge line of the object after moving to the second destination.

[0039]

[0040] According to at least one of the embodiments of the present disclosure, the robot can safely move to a location where the service recipient can comfortably receive the service, thereby improving usability and reliability.

[0041] According to at least one of the embodiments of the present disclosure, it is possible to implement optimal obstacle recognition and driving technology that takes into account the characteristics of the sensors equipped with the robot.

[0042] According to at least one of the embodiments of the present disclosure, when a robot provides a serving service in a restaurant, it can approach the table as close as possible to the table and stop smoothly within a range where a collision does not occur even in a situation where there is a positional error or the table has moved.

[0043] According to at least one of the embodiments of the present disclosure, a robot can be provided that is highly efficient and reliable even in an environment with many changing situations by effectively establishing a path and driving method according to a situation and driving.

[0044] Figure 1 illustrates an AI device according to one embodiment of the present disclosure.

[0045] FIG. 2 illustrates an AI server according to one embodiment of the present disclosure.

[0046] FIG. 3 illustrates an AI system according to one embodiment of the present disclosure.

[0047] Figure 4 illustrates a robot according to an embodiment of the present disclosure.

[0048] FIG. 5 is a block diagram illustrating a control relationship between major components of a robot according to an embodiment of the present disclosure.

[0049] Figures 6a to 6c illustrate situations that may occur when providing services using a robot.

[0050] FIG. 7 is a flowchart illustrating a method of operating a robot according to an embodiment of the present disclosure.

[0051] FIGS. 8 to 16 are drawings for reference in the description of the method of operating a robot according to an embodiment of the present disclosure.

[0052]

[0053] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. However, the present disclosure is not limited to these embodiments and can be modified in various forms.

[0054] In the drawings, parts that are not related to the description are omitted in order to clearly and concisely explain the present disclosure, and the same drawing reference numerals are used for the same or extremely similar parts throughout the specification.

[0055] Meanwhile, the suffixes "module" and "part" used in the following description are given solely for the convenience of writing this specification, and do not impart any particularly significant meaning or role to them. Therefore, the terms "module" and "part" may be used interchangeably.

[0056] Additionally, while terms such as "first" and "second" may be used in this specification to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another.

[0057] Artificial Intelligence (AI) refers to the field of study that studies artificial intelligence or the methodologies for creating it, while Machine Learning (ML) refers to the field that defines various problems in the field of AI and studies the methodologies for solving them. Machine learning is also defined as an algorithm that improves performance on a task through consistent experience.

[0058] An artificial neural network (ANN) is a model used in machine learning. It can refer to a model with problem-solving capabilities, comprised of artificial neurons (nodes) formed by the connection of synapses. An ANN can be defined by the connection patterns between neurons in different layers, the learning process that updates model parameters, and the activation function that generates output values.

[0059] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer contains one or more neurons, and the artificial neural network may include synapses connecting neurons. In an artificial neural network, each neuron can output a function value of an activation function based on input signals, weights, and biases received through the synapses.

[0060] Model parameters are parameters determined through learning, including synaptic connection weights and neuron biases. Hyperparameters are parameters that must be set before learning in machine learning algorithms, including the learning rate, number of iterations, mini-batch size, and initialization function.

[0061] The goal of artificial neural network training can be seen as determining model parameters that minimize a loss function. The loss function can be used as an indicator for determining optimal model parameters during the artificial neural network training process.

[0062] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

[0063] Supervised learning refers to a method for training an artificial neural network when given labels for the training data. The labels can refer to the correct answer (or output value) that the artificial neural network must infer when the training data is input to the artificial neural network. Unsupervised learning can refer to a method for training an artificial neural network when the training data is not given labels. Reinforcement learning can refer to a learning method in which an agent defined within a given environment is trained to select actions or action sequences that maximize the cumulative reward in each state.

[0064] Machine learning implemented with a deep neural network (DNN) containing multiple hidden layers among artificial neural networks is also called deep learning, and deep learning is a subset of machine learning. Hereinafter, the term "machine learning" is used to encompass deep learning.

[0065] Object detection models using machine learning include the single-stage YOLO (You Only Look Once) model and the two-stage Faster R-CNN (Regions with Convolution Neural Networks) model.

[0066] The YOLO (You Only Look Once) model is a model that can predict objects and their locations within an image by looking at the image only once.

[0067] The YOLO (You Only Look Once) model divides the original image into grids of equal size. For each grid, it predicts the number of bounding boxes in a predefined shape centered around the grid center, and calculates confidence based on this.

[0068] Afterwards, whether the image contains an object or is just a background is included, and locations with high object confidence are selected so that the object category can be identified.

[0069] The Faster R-CNN (Regions with Convolution Neural Networks) model is a model that can detect objects faster than the RCNN model and the Fast RCNN model.

[0070] This article specifically explains the Faster R-CNN (Regions with Convolution Neural Networks) model.

[0071] First, feature maps are extracted from the image using a Convolution Neural Network (CNN) model. Based on the extracted feature maps, multiple regions of interest (RoIs) are extracted. RoI pooling is performed for each region of interest.

[0072] RoI pooling is a process of setting a grid to a predetermined size of H x W for the feature map onto which the region of interest is projected, extracting the largest value for each cell included in each grid, and extracting a feature map with a size of H x W.

[0073] A feature vector is extracted from a feature map having a size of H x W, and object identification information can be obtained from the feature vector.

[0074] A robot can be defined as a machine that automatically performs or operates a given task based on its own capabilities. Specifically, a robot capable of perceiving its environment, making independent judgments, and performing actions can be called an intelligent robot.

[0075] Robots can be classified into industrial, medical, household, and military types depending on their purpose or field of use.

[0076] Robots are equipped with actuators or motors, enabling them to perform various physical actions, such as moving robot joints. Furthermore, mobile robots include wheels, brakes, propellers, and other actuators within their actuators, enabling them to move on the ground or fly in the air.

[0077] Self-driving refers to the technology of driving on one's own, and an autonomous vehicle refers to a vehicle that drives without user intervention or with minimal user intervention.

[0078] For example, autonomous driving can include technologies that maintain the driving lane, technologies that automatically adjust speed such as adaptive cruise control, technologies that automatically drive along a set route, and technologies that automatically set a route and drive when a destination is set.

[0079] Vehicles include vehicles equipped only with internal combustion engines, hybrid vehicles equipped with both internal combustion engines and electric motors, and electric vehicles equipped only with electric motors, and may include not only automobiles but also trains, motorcycles, etc.

[0080] At this time, autonomous vehicles can be viewed as robots with autonomous driving functions.

[0081] FIG. 1 illustrates an AI device (100) according to one embodiment of the present disclosure.

[0082] The AI ​​device (100) can be implemented as a fixed device or a movable device, such as a TV, a projector, a mobile phone, a smart phone, a desktop computer, a laptop, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, digital signage, a robot, a vehicle, etc.

[0083] Referring to FIG. 1, the terminal (100) may include a communication interface (110), an input interface (120), a running processor (130), a sensing unit (140), an output interface (150), a memory (170), and a processor (180).

[0084] The communication interface (110) can transmit and receive data with external devices such as other AI devices (100a to 100e) or AI servers (200) using wired or wireless communication technology. For example, the communication interface (110) can transmit and receive sensor information, user input, learning models, control signals, etc. with external devices.

[0085] At this time, the communication technologies used by the communication interface (110) include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth (Bluetooth™), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0086] The input interface (120) can obtain various types of data.

[0087] At this time, the input interface (120) may include a camera for inputting a video signal, a microphone for receiving an audio signal, a user input interface for receiving information from a user, etc. Here, the camera or microphone may be treated as a sensor, and the signal obtained from the camera or microphone may be referred to as sensing data or sensor information.

[0088] The input interface (120) can obtain input data to be used when obtaining output using learning data and learning models for model learning. The input interface (120) can also obtain unprocessed input data, in which case the processor (180) or learning processor (130) can extract input features as preprocessing for the input data.

[0089] The learning processor (130) can train a model composed of an artificial neural network using learning data. Here, the trained artificial neural network may be referred to as a learning model. The learning model can be used to infer result values ​​for new input data other than the learning data, and the inferred values ​​can be used as a basis for making decisions regarding certain actions.

[0090] At this time, the running processor (130) can perform AI processing together with the running processor (240) of the AI ​​server (200).

[0091] At this time, the running processor (130) may include a memory integrated or implemented in the AI ​​device (100). Alternatively, the running processor (130) may be implemented using a memory (170), an external memory directly coupled to the AI ​​device (100), or a memory maintained in an external device.

[0092] The sensing unit (140) can obtain at least one of internal information of the AI ​​device (100), information about the surrounding environment of the AI ​​device (100), and user information using various sensors.

[0093] At this time, the sensors included in the sensing unit (140) include a proximity sensor, a light sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, a lidar, a radar, etc.

[0094] The output interface (150) can generate output related to visual, auditory, or tactile sensations.

[0095] At this time, the output interface (150) may include a display unit that outputs visual information, a speaker that outputs auditory information, a haptic module that outputs tactile information, etc.

[0096] The memory (170) can store data that supports various functions of the AI ​​device (100). For example, the memory (170) can store input data, learning data, learning models, learning history, etc. obtained from the input interface (120).

[0097] The processor (180) may determine at least one executable operation of the AI ​​device (100) based on information determined or generated using a data analysis algorithm or a machine learning algorithm. Then, the processor (180) may control components of the AI ​​device (100) to perform the determined operation.

[0098] To this end, the processor (180) can request, retrieve, receive or utilize data from the running processor (130) or memory (170), and control components of the AI ​​device (100) to execute at least one of the executable operations, a predicted operation or an operation determined to be desirable.

[0099] At this time, if the processor (180) requires connection of an external device to perform a determined operation, it can generate a control signal for controlling the external device and transmit the generated control signal to the external device.

[0100] The processor (180) can obtain intent information for user input and determine the user's requirements based on the obtained intent information.

[0101] At this time, the processor (180) can obtain intent information corresponding to the user input by using at least one of an STT (Speech To Text) engine for converting voice input into a string or a natural language processing (NLP) engine for obtaining intent information of natural language.

[0102] At this time, at least one of the STT engine or the NLP engine may be configured with an artificial neural network, at least in part, trained according to a machine learning algorithm. Furthermore, at least one of the STT engine or the NLP engine may be trained by the learning processor (130), the learning processor (240) of the AI ​​server (200), or through distributed processing thereof.

[0103] The processor (180) can collect history information including the operation details of the AI ​​device (100) or the user's feedback on the operation, and store the information in the memory (170) or the learning processor (130), or transmit the information to an external device such as an AI server (200). The collected history information can be used to update the learning model.

[0104] The processor (180) can control at least some of the components of the AI ​​device (100) to drive an application program stored in the memory (170). Furthermore, the processor (180) can operate two or more of the components included in the AI ​​device (100) in combination to drive the application program.

[0105] FIG. 2 illustrates an AI server (200) according to one embodiment of the present disclosure.

[0106] Referring to FIG. 2, the AI ​​server (200) may refer to a device that trains an artificial neural network using a machine learning algorithm or utilizes a trained artificial neural network. Here, the AI ​​server (200) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network. In this case, the AI ​​server (200) may be included as part of the AI ​​device (100) and may perform at least a portion of the AI ​​processing.

[0107] The AI ​​server (200) may include a communication interface (210), memory (230), a learning processor (240), and a processor (260).

[0108] The communication interface (210) can transmit and receive data with an external device such as an AI device (100).

[0109] The memory (230) may include a model storage (231). The model storage (231) may store a model (or artificial neural network, 231a) being learned or learned through a learning processor (240).

[0110] A learning processor (240) can train an artificial neural network (231a) using learning data. The learning model can be used while mounted on the AI ​​server (200) of the artificial neural network, or can be mounted on an external device such as an AI device (100).

[0111] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (230).

[0112] The processor (260) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0113] Figure 3 illustrates an AI system (1) according to one embodiment of the present disclosure.

[0114] Referring to FIG. 3, an AI system (1) is connected to a cloud network (10) by at least one of an AI server (200), a robot (100a), an autonomous vehicle (100b), an XR device (100c), a smartphone (100d), or an appliance (100e). Here, a robot (100a), an autonomous vehicle (100b), an XR device (100c), a smartphone (100d), or an appliance (100e) to which AI technology is applied may be referred to as an AI device (100a to 100e).

[0115] A cloud network (10) may refer to a network that constitutes part of a cloud computing infrastructure or exists within a cloud computing infrastructure. Here, the cloud network (10) may be configured using a 3G network, a 4G or LTE (Long Term Evolution) network, a 5G network, etc.

[0116] That is, each device (100a to 100e, 200) constituting the AI ​​system (1) can be connected to each other via a cloud network (10). In particular, each device (100a to 100e, 200) can communicate with each other via a base station, but can also communicate with each other directly without going through a base station.

[0117] The AI ​​server (200) may include a server that performs AI processing and a server that performs operations on big data.

[0118] The AI ​​server (200) is connected to at least one of the AI ​​devices constituting the AI ​​system (1), such as a robot (100a), an autonomous vehicle (100b), an XR device (100c), a smartphone (100d), or a home appliance (100e), through a cloud network (10), and can assist at least part of the AI ​​processing of the connected AI devices (100a to 100e).

[0119] At this time, the AI ​​server (200) can train an artificial neural network according to a machine learning algorithm on behalf of the AI ​​devices (100a to 100e), and can directly store the learning model or transmit it to the AI ​​devices (100a to 100e).

[0120] At this time, the AI ​​server (200) can receive input data from the AI ​​devices (100a to 100e), infer a result value for the received input data using a learning model, and generate a response or control command based on the inferred result value and transmit it to the AI ​​devices (100a to 100e).

[0121] Alternatively, the AI ​​device (100a to 100e) may infer a result value for input data using a direct learning model and generate a response or control command based on the inferred result value.

[0122] Below, various embodiments of AI devices (100a to 100e) to which the above-described technology is applied are described. Here, the AI ​​devices (100a to 100e) illustrated in FIG. 3 can be viewed as specific embodiments of the AI ​​device (100) illustrated in FIG. 1.

[0123] The robot (100a) can be implemented as a guide robot, transport robot, cleaning robot, wearable robot, entertainment robot, pet robot, unmanned flying robot, etc. by applying AI technology.

[0124] The robot (100a) may include a robot control module for controlling movement, and the robot control module may mean a software module or a chip that implements the same in hardware.

[0125] The robot (100a) can obtain status information of the robot (100a), detect (recognize) the surrounding environment and objects, generate map data, determine a movement path and driving plan, determine a response to user interaction, or determine an action using sensor information obtained from various types of sensors.

[0126] Here, the robot (100a) can use sensor information acquired from at least one sensor among lidar, radar, and camera to determine a movement path and driving plan.

[0127] The robot (100a) can perform the above-described operations using a learning model comprised of at least one artificial neural network. For example, the robot (100a) can recognize its surroundings and objects using the learning model, and determine operations using the recognized surrounding environment information or object information. Here, the learning model may be learned directly by the robot (100a) or by an external device such as an AI server (200).

[0128] At this time, the robot (100a) may perform an action by generating a result using a direct learning model, but may also perform an action by transmitting sensor information to an external device such as an AI server (200) and receiving the result generated accordingly.

[0129] The robot (100a) can determine a movement path and a driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and control a driving unit to drive the robot (100a) according to the determined movement path and driving plan.

[0130] Map data may include object identification information for various objects positioned in the space where the robot (100a) moves. For example, map data may include object identification information for fixed objects such as walls and doors, as well as movable objects such as flower pots and desks. Furthermore, object identification information may include name, type, distance, location, etc.

[0131] Additionally, the robot (100a) can perform actions or drive by controlling the driving unit based on the user's control / interaction. At this time, the robot (100a) can acquire intention information regarding the interaction based on the user's actions or voice utterances, and determine a response based on the acquired intention information to perform the action.

[0132] An autonomous vehicle (100b) can be implemented as a mobile robot, vehicle, or unmanned aerial vehicle by applying AI technology.

[0133] The autonomous vehicle (100b) may include an autonomous driving control module for controlling autonomous driving functions. The autonomous driving control module may refer to a software module or a chip implementing the same as hardware. The autonomous driving control module may be included internally as a component of the autonomous vehicle (100b), but may also be configured as separate hardware and connected to the exterior of the autonomous vehicle (100b).

[0134] An autonomous vehicle (100b) can obtain status information of the autonomous vehicle (100b), detect (recognize) the surrounding environment and objects, generate map data, determine a movement path and driving plan, or determine an action using sensor information obtained from various types of sensors.

[0135] Here, the autonomous vehicle (100b) can use sensor information acquired from at least one sensor among lidar, radar, and camera, similar to the robot (100a), to determine a movement path and driving plan.

[0136] In particular, the autonomous vehicle (100b) can recognize the environment or objects in an area where the field of view is obstructed or an area beyond a certain distance by receiving sensor information from external devices, or can receive information recognized directly from external devices.

[0137] The autonomous vehicle (100b) can perform the above-described operations using a learning model comprised of at least one artificial neural network. For example, the autonomous vehicle (100b) can recognize its surroundings and objects using the learning model, and determine a driving route using the recognized surrounding environment information or object information. Here, the learning model may be learned directly by the autonomous vehicle (100b) or by an external device such as an AI server (200).

[0138] At this time, the autonomous vehicle (100b) may perform an action by generating a result using a direct learning model, but may also perform an action by transmitting sensor information to an external device such as an AI server (200) and receiving the result generated accordingly.

[0139] An autonomous vehicle (100b) can determine a movement path and a driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and control a driving unit to drive the autonomous vehicle (100b) according to the determined movement path and driving plan.

[0140] Map data may include object identification information for various objects located in the space (e.g., a road) where the autonomous vehicle (100b) travels. For example, map data may include object identification information for fixed objects such as streetlights, rocks, and buildings, as well as movable objects such as vehicles and pedestrians. Furthermore, object identification information may include name, type, distance, location, and the like.

[0141] Additionally, the autonomous vehicle (100b) can perform actions or drive by controlling the driving unit based on the user's control / interaction. At this time, the autonomous vehicle (100b) can acquire intention information regarding the interaction based on the user's actions or voice utterances, and determine a response based on the acquired intention information to perform the action.

[0142] The robot (100a) can be implemented as a guide robot, transport robot, cleaning robot, wearable robot, entertainment robot, pet robot, unmanned flying robot, etc. by applying AI technology and autonomous driving technology.

[0143] A robot (100a) to which AI technology and autonomous driving technology are applied may refer to a robot itself with autonomous driving function, or a robot (100a) that interacts with an autonomous vehicle (100b).

[0144] A robot (100a) with an autonomous driving function can be a general term for devices that move on their own along a given path without user control or move by determining the path on their own.

[0145] A robot (100a) with autonomous driving capabilities and an autonomous vehicle (100b) may use a common sensing method to determine one or more of a movement path or a driving plan. For example, a robot (100a) with autonomous driving capabilities and an autonomous vehicle (100b) may use information sensed via lidar, radar, and cameras to determine one or more of a movement path or a driving plan.

[0146] A robot (100a) interacting with an autonomous vehicle (100b) may exist separately from the autonomous vehicle (100b), and may be linked to autonomous driving functions within the autonomous vehicle (100b) or perform actions linked to a user riding in the autonomous vehicle (100b).

[0147] At this time, the robot (100a) interacting with the autonomous vehicle (100b) can control or assist the autonomous driving function of the autonomous vehicle (100b) by acquiring sensor information on behalf of the autonomous vehicle (100b) and providing it to the autonomous vehicle (100b), or by acquiring sensor information and generating surrounding environment information or object information and providing it to the autonomous vehicle (100b).

[0148] Alternatively, a robot (100a) interacting with an autonomous vehicle (100b) may monitor a user riding in the autonomous vehicle (100b) or control functions of the autonomous vehicle (100b) through interaction with the user. For example, if the robot (100a) determines that the driver is drowsy, it may activate the autonomous driving function of the autonomous vehicle (100b) or assist in controlling the driving unit of the autonomous vehicle (100b). Here, the functions of the autonomous vehicle (100b) controlled by the robot (100a) may include not only the autonomous driving function, but also functions provided by a navigation system or audio system installed inside the autonomous vehicle (100b).

[0149] Alternatively, a robot (100a) interacting with an autonomous vehicle (100b) may provide information to the autonomous vehicle (100b) or assist functions from outside the autonomous vehicle (100b). For example, the robot (100a) may provide traffic information, including signal information, to the autonomous vehicle (100b), such as a smart traffic light, or may interact with the autonomous vehicle (100b) to automatically connect an electric charger to a charging port, such as an automatic electric charger for an electric vehicle.

[0150] FIG. 4 illustrates an AI device (100) according to an embodiment of the present disclosure. More specifically, FIG. 4 illustrates a robot (100a) according to an embodiment of the present disclosure.

[0151] Fig. 4 illustrates serving robots (100a1, 100a2) that primarily serve items such as food. Referring to Fig. 4, the robot (100a) may include a main body (400) and a service module (490).

[0152] The main body (400) may form the base of the robot (100a). The main body (400) may be equipped with a driving wheel (402) that drives the robot (100a). The driving wheel (402) may protrude from the lower side of the main body (400).

[0153] The main body (400) can accommodate various components. The main body (400) can be equipped with a plurality of sensors.

[0154] For example, the main body (400) may be equipped with a lidar sensor (420). The lidar sensor (420) may be placed in a recessed portion formed in the main body (400).

[0155] Additionally, the main body (400) may be equipped with a plurality of ToF (time of flight) sensors (430). The plurality of ToF sensors (430) may be spaced apart from each other in the circumferential direction of the main body (400). More specifically, the ToF sensors (430) are arranged on the outer circumference of the main body (400) to detect objects around the robot (100a).

[0156] The ToF sensor (430) may be installed at a lower position than the lidar sensor (420). More specifically, the height of the ToF sensor (430) may be lower than the height of the lidar sensor (420) based on the lower end of the main body (400).

[0157] The ToF sensor (430) may be an optical ToF sensor (430) that calculates distance by measuring the time (time of flight) that light takes to hit an object and reflect. Alternatively, the ToF sensor (430) may use an ultrasonic sensor that measures ToF (time of flight) using ultrasonic waves.

[0158] A camera module (410) may be placed in the main body (400). The camera module (410) may include one or more cameras. The camera module (410) may include a camera sensor (412) for acquiring images and a depth camera (411) capable of measuring the distance to an obstacle.

[0159] The depth camera (411) can acquire depth images and measure the distance to obstacles. The camera sensor (412) can be an RGB 2D camera and can acquire images of the surroundings of the robot (100a).

[0160] Unlike the example of FIG. 4, the depth camera (411) and the camera sensor (412) may be individually positioned at different locations. The camera module (410) may be an RGB-D camera in which the depth camera (411) and the camera sensor (412) are integrated.

[0161] The service module (490) can cover the main body (400) from the upper side. The service module (490) includes a plurality of shelves and can accommodate items. The robot (100a) can autonomously drive while items are accommodated in the service module (490).

[0162] The robot (100a) according to the embodiment of the present disclosure is not limited to the example of FIG. 4. The serving robot (100a1) illustrated in FIG. 4 (a) and the serving robot (100a2) illustrated in FIG. 4 (b) differ in the shapes of the main body (400) and the service module (490), the sensors (410, 420, 430), and the positions of the driving wheels (402). In this way, the robot (100a) according to the embodiment of the present disclosure can be configured in various ways in terms of the appearance, the positions of each component, and the combination thereof.

[0163] Fig. 5 is a block diagram illustrating the control relationship between the main components of a robot (100a) according to one embodiment of the present disclosure. Descriptions overlapping with Fig. 1 are omitted.

[0164] Referring to FIG. 5, a robot (100a) according to an embodiment of the present disclosure includes a processor (180) that controls the overall operation of the robot (100a), a memory (170) that stores various data, a driving unit (190) that moves a main body (400) including a driving wheel (402), a sensing unit (140) that includes a plurality of sensors, and an input interface (120). In addition, the robot (100a) further includes the above-described learning processor (130) and can utilize AI for obstacle recognition, etc.

[0165] The processor (180) can control the overall operation of the robot (100a) by controlling the memory (170), driving unit (190), sensing unit (140), etc. within the robot (100a).

[0166] The memory (170) records various information necessary for controlling the robot (100a) and may include a volatile or non-volatile recording medium.

[0167] A map of the driving area may be stored in the memory (170). The map may be input by an external terminal, server, etc. that can exchange information with the robot (100a) via wired or wireless communication, or may be generated by the robot (100a) through self-learning.

[0168] The map for the driving area stored in the memory (170) may be a navigation map used for driving, a SLAM (Simultaneous localization and mapping) map used for location recognition, a global location map used for global location recognition, a local map used for local location recognition, an obstacle recognition map in which information about recognized obstacles is recorded, etc.

[0169] A global path refers to the path from the robot's starting or current location to the target point. A local path refers to a newly created path that the robot creates while driving along the global path or a newly created local path. This path is created by the robot by detecting surrounding obstacles at regular intervals.

[0170] Meanwhile, as described above, maps can be stored and managed in the memory (170) by purpose. However, the maps may not be clearly divided by purpose. Multiple pieces of information can be stored in a single map so that it can be used for at least two purposes. For example, a local map may include information about obstacles.

[0171] The driving unit (190) can move the main body (400) under the control of the processor (180). The driving unit (190) may include at least one driving wheel (402) that moves the main body of the robot (100a). The driving unit (190) may include a driving motor (not shown) that is connected to the driving wheel (402) and rotates the driving wheel (402).

[0172] Driving wheels (402) may be provided on the left and right sides of the main body (400), and are hereinafter referred to as left and right wheels, respectively. The left and right wheels may be driven by a single driving motor, but a left-wheel driving motor for driving the left wheel and a right-wheel driving motor for driving the right wheel may be provided, as needed. By differentiating the rotational speeds of the left and right wheels, the driving direction of the main body (400) can be changed to the left or right.

[0173] The robot (100a) may include a sensing unit (140) that includes sensors that sense various data related to the operation and status of the robot (100a).

[0174] The robot (100a) may include a motion detection sensor (141) that detects the motion of the robot (100a) and outputs motion information. For example, a gyro sensor, a wheel sensor, an acceleration sensor, etc. may be used as the motion detection sensor.

[0175] The gyro sensor detects the direction of rotation and the angle of rotation when the robot (100a) moves according to the driving mode. The gyro sensor detects the angular velocity of the robot (100a) and outputs a voltage value proportional to the angular velocity. The processor (180) calculates the direction of rotation and the angle of rotation using the voltage value output from the gyro sensor.

[0176] The wheel sensor is connected to the left and right wheels and detects the rotational speed of the wheels. Here, the wheel sensor may be a rotary encoder. The rotary encoder detects and outputs the rotational speed of the left and right wheels.

[0177] The processor (180) can calculate the rotational speed of the left and right wheels using the rotational speed. In addition, the processor (180) can calculate the rotational angle using the difference in the rotational speed of the left and right wheels.

[0178] The acceleration sensor detects changes in the robot's (100a) speed, such as changes in the robot's (100a) speed due to starting, stopping, changing direction, or colliding with an object. The acceleration sensor is attached to a location adjacent to the main wheel or auxiliary wheel, and can detect wheel slippage or idling.

[0179] Additionally, the acceleration sensor can detect changes in the speed of the robot (100a). That is, the acceleration sensor detects the amount of impact resulting from the change in speed and outputs a corresponding voltage value. Therefore, the acceleration sensor can function as an electronic bumper.

[0180] The processor (180) can calculate changes in the position of the robot (100a) based on motion information output from the motion detection sensor. This position corresponds to an absolute position using image information, and thus becomes a relative position. Through this relative position recognition, the robot (100a) can improve the performance of position recognition using image information and obstacle information.

[0181] The motion detection sensor (141) may include an IMU (Inertial Measurement Unit) sensor. The IMU sensor can recognize the movement status of the robot (100a) using an accelerometer, a gyrometer, a magnetometer, and an altimeter.

[0182] The robot (100a) may include a camera sensor (412) capable of capturing a predetermined range. The camera sensor (412) captures the surroundings of the robot (100a), the external environment, etc., and multiple camera sensors (412) may be installed in each area to improve capturing efficiency.

[0183] The processor (180) can judge external situations or recognize users, obstacles, etc. based on images captured by the camera sensor (412).

[0184] Additionally, the processor (180) can control the robot (100a) to drive based on an image captured by the camera sensor (412).

[0185] Meanwhile, the image captured by the camera sensor (412) can be stored in the memory (170).

[0186] The robot (100a) may include an obstacle detection sensor (143) that detects obstacles. The obstacle detection sensor (143) may include an infrared sensor, an ultrasonic sensor, a ToF sensor (430), an RF sensor, a geomagnetic sensor, a PSD (Position Sensitive Device) sensor, a cliff detection sensor that detects the presence of a cliff on the ground within the driving area, a Lidar (light detection and ranging) sensor (420), etc.

[0187] The obstacle detection sensor (143) can measure not only the presence of an obstacle but also the distance between the obstacle and the robot (100a). Therefore, the obstacle detection sensor (143) may also be referred to as a distance sensor.

[0188] Meanwhile, the obstacle detection sensor (143) detects objects, particularly obstacles, present in the driving (moving) direction of the robot (100a) and transmits obstacle information to the processor (180). At this time, the processor (180) can control the movement of the robot (100a) according to the location of the detected obstacle.

[0189] Meanwhile, the lidar sensor (420) outputs a laser and provides information such as the distance, position, direction, and material of an object that reflects the laser, and can acquire topographic information of the driving area. The robot (100a) can acquire 360-degree topographic (geometry) information using the lidar sensor (420).

[0190] The robot (100a) can generate a map by identifying the distance, location, direction, etc. of objects sensed by the lidar sensor (420). The robot (100a) can obtain topographic information of the driving area by analyzing the laser reception pattern, such as the time difference or signal intensity of the laser reflected from the outside and received. In addition, the robot (100a) can generate a map using the topographic information acquired through the lidar sensor (420).

[0191] For example, the robot (100a) can perform a lidar slam to recognize the current location by comparing the surrounding terrain information acquired at the current location through the lidar sensor (420) with previously stored terrain information based on the lidar sensor or by comparing the acquired terrain information.

[0192] The processor (180) can receive a destination through the user input interface of the input interface (120). Alternatively, the processor (180) can receive the destination through the communication interface (110). The processor (180) can generate a driving path of the robot (100a). When the processor (180) receives a destination, it can load a map stored in the memory (170). For example, the user can select a destination, such as in front of a table, in front of a chair, or a specific location, which are predetermined within the map. Accordingly, the processor (180) can convert the destination selected by the user into coordinates corresponding to the map and generate a driving path from the starting point, which is the current location, to the arrival point corresponding to the destination.

[0193] Meanwhile, the map stored in the memory (170) may be a grid map including a plurality of grids for the driving area. The grid map includes a plurality of grid cells formed by a plurality of vertical grid lines and a plurality of horizontal grid lines. A grid cell is the smallest unit formed by four grid lines and may also be referred to as a grid.

[0194] Additionally, a map containing obstacle information may be a cost map in which costs are assigned to multiple grids. For example, a global map used for global path planning and / or a local map used for local path planning may include a cost map.

[0195] The processor (180) can generate a map of the driving area based on information detected by the sensing unit (140). The processor (180) can generate a two-dimensional or three-dimensional map by fusing data acquired from various sensors of the sensing unit (140).

[0196] The processor (180) can assign a cost to each grid included in the cost map based on obstacle information detected by the sensing unit (140) and generate a driving route with the lowest cost from the starting point to the destination point.

[0197] Additionally, the processor (180) can update the cost map to reflect dynamic obstacles detected during driving and newly detected fixed obstacles. Furthermore, the processor (180) can regenerate the route to the destination based on the updated cost map.

[0198] Meanwhile, when a robot (100a) provides a serving service, delivering food or goods to a restaurant, store, etc., it is required that the robot (100a) approach the table or person as closely as possible. For example, upon arriving at a customer's table, the robot (100a) should be within arm's reach of the customer.

[0199] Figures 6a to 6c illustrate situations that may occur when providing services using a robot.

[0200] When a robot (100a) provides a food delivery service in a restaurant, customers and restaurant owners often request that the robot (100a) be positioned as close to the table as possible. For the convenience of customers, restaurants require the robot (100a) to approach the table as close as possible and then stop upon arrival.

[0201] Accordingly, the destination is set to a location close to the table, and the robot (100a) autonomously drives to the destination and serves food, etc.

[0202] FIG. 6a illustrates a destination point (630) set on a map for delivering serving items to a table (610). When a movement command is received with the table (610) or the destination point (630) corresponding to the table (610) as the destination, the processor (180) generates a driving path to the destination point (630), and the robot (100a) drives along the generated driving path to the destination location (630).

[0203] Meanwhile, the distance between the arrival point (630) of the robot (100a) and the customer (640) is a set appropriate distance (d1), so that the customer (640) can receive the served food with minimal movement.

[0204] The destination point (630) can be set as close as possible to the table (610). However, depending on the movement of the table (610) or the position recognition error of the robot (100a), the destination point (630) may not be fixed and a problem may occur.

[0205] Referring to FIG. 6b, restaurant employees, customers, etc. may move the table (610) away from the fixed destination point (630). In this case, even if the robot (100a) accurately arrives at the destination point (630) set corresponding to the table (610), the distance (d2) between the robot (100a) and the customer (640) increases, making it inconvenient for the customer (640) to receive food.

[0206] Referring to FIG. 6c, restaurant employees, customers, etc. may move the table (610) toward a fixed destination point (630) or move above the destination point (630). In this case, when the robot (100a) moves to the destination point (630), the robot (100a) may collide with the table (610), or the robot (100a) may move back and forth around the table (610) to avoid collision, resulting in a long delivery time.

[0207] Additionally, the robot (100a) may collide with the table (610) while rotating toward the table (610) for serving after arriving at a point of interest (POI) close to the destination. Additionally, if a collision occurs with the robot (100a) while serving, hot food or dangerous objects such as scissors may fall on the customer (640).

[0208] As described with reference to FIGS. 6a to 6c, when using a fixed arrival point (630), the arrival position of the robot (100a) may not be constant.

[0209] The same situation may also occur due to an error in position recognition of the robot (100a).

[0210] For example, the conventional method of processing arrival at the destination point (630) set on the map is reflected as a table arrival location error if there is an error in the Slam Map or a Slam location error occurs.

[0211] However, SLAM is inherently prone to errors. For example, the lidar-centric fslim method can introduce errors due to changes in surrounding structures, glass walls that the lidar cannot detect, and moving obstacles. Furthermore, the vslam method, which relies on images acquired from cameras, can introduce errors due to surrounding structures, brightness changes, and moving obstacles. Furthermore, methods using ceiling-mounted markers can introduce errors due to inaccessible areas and insufficient density of markers.

[0212] Accordingly, the present disclosure proposes a method for improving customer convenience and preventing safety accidents by allowing a robot (100a) to arrive as close as possible to a specific object (e.g., a table (610)) within a range where the robot (100a) does not collide with it.

[0213] When a robot (100a) provides a serving service in a restaurant, even if an error occurs in position recognition or the table is moved, it can safely move to the closest position without colliding with the object by moving to a destination set at a predetermined distance from an object such as a table and then approaching the object.

[0214] In addition, the robot (100a) can drive effectively by first moving to a virtual destination based on a fixed destination on the map, then moving to the final destination, and changing the driving mode according to the driving situation.

[0215] The robot (100a) can receive a command to move to a predetermined destination through the input interface (1120). Alternatively, the robot (100a) can also receive a command to move to a predetermined destination through the communication interface (1110).

[0216] Depending on the driving area and service environment, destinations can be set to correspond to objects. For example, in a restaurant, multiple destinations can be set to correspond to each table. Information about the set destinations can be registered on a map stored in memory (170).

[0217] Meanwhile, the robot (100a) may receive a command to move to a first destination corresponding to a designated object via the input interface (1120) or the communication interface (1110). The designated object may be a table. In this case, the first destination may be set at a location a suitable distance (d1) from the table. The first destination may be a destination fixedly stored on the map.

[0218] The processor (180) sets a second destination that is spaced apart from the first destination so that the robot (100a) does not move directly to the first destination. The processor (180) sets the second destination so that the distance to the object is longer than that of the first destination. In other words, the distance between the second destination and the object is longer than the distance between the first destination and the object.

[0219] After setting the second destination, the processor (180) controls the robot (100a) to move to the second destination. Since the second destination is further away from the object than the first destination, collision can be prevented even if the object moves toward the first destination.

[0220] Meanwhile, when the robot (100a) arrives at the second destination, the processor (180) controls it to drive close to the object.

[0221] When a robot (100a) provides service in a restaurant with multiple tables, each of the multiple tables may correspond to a designated object. Furthermore, the processor (180) may control the robot to sequentially perform proximity driving to approach the object after automatically moving to the second destination, rather than directly to the first destination.

[0222] When the destination is a table, the processor (180) can control the vehicle to set a second destination away from the destination instead of going to the destination at once, and to drive in close proximity from the second destination to the table.

[0223] Meanwhile, the second destination may be a new destination generated by the processor (180) during driving. That is, the processor (180) can dynamically set a new second destination for each drive. This allows for better adaptation to environmental changes, such as changes in the spacing between tables.

[0224] The memory (170) may store coordinate information and direction information of the first destination. The coordinate information may include location information of the first destination on the map, and the direction information may include angle information indicating where the robot (100a) should look.

[0225] The coordinates of the second destination may be set to be a predetermined distance away from the coordinates of the first destination in the opposite direction in which the object is located. The direction of the second destination may be set in the vertical direction of the edge line of the object.

[0226] The processor (180) can calculate the opposite position of an object, such as a table, based on the position and angle information of the set first destination, and dynamically change the position to a second destination, which is a temporary arrival position.

[0227] Under the control of the processor (180), the robot (100a) approaches the object from the second destination as close as possible to a line without risk of collision based on sensor information. Accordingly, the robot (100a) can stop while maintaining a constant distance from the object. Even if an error occurs in the robot's (100a) position recognition or the table moves slightly, the robot (100a) can stop while maintaining a constant distance from the table.

[0228] Proximity driving may be driving in which the robot (100a) approaches an object while moving straight toward the object from a second destination. The processor (180) may control the robot (100a) to approach the object at the shortest distance without risk of collision based on sensing data from the obstacle detection sensor (143).

[0229] According to an embodiment, the robot (100a) can, under the control of the processor (180), approach from the second destination to the first destination as close as possible to a line without risk of collision based on sensor information.

[0230] Meanwhile, the processor (180) may determine a second destination based on the sensing area range of the depth camera (411). The second destination may be set by considering a blind area where the depth camera (411) can no longer see the object, a location where sensing accuracy begins to decline, etc. For example, if the depth camera (411) has a blind area within 0.3 m and sensing accuracy declines at 1 m or more, the second destination may be set within a range of 0.3 m to 1 m from the object.

[0231] Meanwhile, the second destination can be set to a position and direction where the depth camera (411) can see the edge of the object.

[0232] The processor (180) can control the robot (100a) to drive straight in the direction of the object while updating the position of the object seen by the depth camera (411) during proximity driving.

[0233] The processor (180) can reset the second destination, if necessary, while updating the location of the object seen by the depth camera (411). In addition, the processor (180) can control the robot (100a) to drive in a straight line from the reset second destination to the object.

[0234] Depending on the driving method, the robot (100a) may require a certain amount of space for movement and direction changes. For example, in the 2-diff wheel method, which changes direction based on the rotational difference between the two wheels, the second destination must be set to take into account the space required for such direction changes.

[0235] Accordingly, the second destination can be set as a location and direction in which the object can be approached by driving by rotation of the driving wheels (402) including the left and right wheels. The processor (180) can set a second destination in which the depth camera (411) can accurately detect the distance to the table and can be approached in close proximity using a 2-diff wheel method.

[0236] Meanwhile, the processor (180) may use different driving modes for driving to the second destination and approaching the object. The processor (180) may control the robot (100a) to drive in the first driving mode to the second destination, and then, when the robot (100a) arrives at the second destination, to approach the object in the second driving mode.

[0237] The first driving mode may be a driving mode that uses SLAM (Simultaneous Localization and Mapping)-based position recognition. That is, the processor (180) may control the robot (100a) to the second destination using a conventional driving method. SLAM-based position recognition may recognize the location of the robot (100a) in a SLAM map using various sensors of the sensing unit (140). However, as described above, fslam, vslam, etc. may have a certain degree of positional error. Therefore, other methods may be more effective for approaching an object as closely as possible.

[0238] Meanwhile, the second driving mode may be a driving mode that uses position recognition based on odometry information.

[0239] Odometry information can be acquired by a motion detection sensor (141). For example, the odometry information may include sensing data from a wheel sensor that detects the rotational speed of a driving wheel (402). The wheel sensor may be an encoder. In addition, the odometry information may include sensing data from an IMU sensor. In addition, the odometry information may be based on sensing data from a wheel sensor and a lidar sensor (420).

[0240] Location recognition based on odometry information can increase errors as the accumulated sensing values ​​accumulate. However, over short distances and short periods of time, location recognition based on odometry information can be more accurate.

[0241] More preferably, the processor (180) can reset the position before the proximity driving start to the reference origin coordinates of the odometry information and control the proximity driving to begin. Accordingly, the processor (180) can more accurately recognize the position of the robot (100a) and control the driving.

[0242] After moving to the second destination, if the direction of the robot (100a) does not match the vertical direction of the edge line of the object, the processor (180) can rotate the robot (100a) to match the direction of the robot (100a) with the vertical direction of the edge line of the object. Thereafter, the robot (100a) can approach the object as much as possible by only driving straight without turning.

[0243] FIG. 7 is a flowchart illustrating an operation method of a robot according to an embodiment of the present disclosure, and FIGS. 8 to 16 are drawings referenced in an explanation of an operation method of a robot according to an embodiment of the present disclosure.

[0244] Referring to FIG. 7, the robot (100a) can receive a user's destination input via an input interface (120) or the like (S710). For example, the user can input a movement command to table poi via the input interface (120). Here, table poi may be the first destination corresponding to a table, which is a designated object.

[0245] Fig. 8 illustrates a fixed destination (820) corresponding to a table (810). Information about the destination (820) may include location information and angle information (821) about the direction the robot (100a) will look after arrival.

[0246] In the past, the processor (180) generated a driving path to the destination (820) at once, and the robot (100a) moved according to the generated driving path.

[0247] According to an embodiment of the present disclosure, when the destination is an object designated as a table (S710), the processor (180) may set a second destination away from the first destination instead of going to the input first destination at once (S720).

[0248] Referring to Fig. 9, a first destination (920) is set corresponding to a table (810). Information about the first destination (920) on the map may include location information and direction information (921). The second destination (910) may be set at a location and direction in which the depth camera (411) can accurately view the corner of the table (810). In addition, the second destination (910) may be set at a location and direction in which approaching the table (810) is possible through 2-diff wheel driving. In addition, the second destination (910) may be set by applying the above-described various conditions in combination.

[0249] The second destination (910) is spaced apart from the table (810). In addition, the second destination (910) may be spaced apart from the table (810) by a predetermined distance in the opposite direction (915) of the direction information (921) of the first destination (920).

[0250] The distance between the second destination (910) and the table (810) is longer than the distance between the first destination (920) and the table (810).

[0251] The robot (100a) can travel to the set second destination (910) using the first method (S730). For example, the robot (100a) can travel to the second destination (910) using the conventional slam method.

[0252] Meanwhile, the robot (100a) can drive from the second destination (910) to the table (810) using a second method (S750). For example, the robot (100a) can approach the table (810) using odometry-based driving rather than slam.

[0253] Referring to FIG. 10, the robot (100a) moves in a normal manner in the first section (tr1) to the second destination (910), and in the second section (tr2) from the second destination (910) toward the table (810), moves straight in a table approaching manner and approaches the table (810).

[0254] Meanwhile, the processor (180) updates the position of the table (810) visible to the depth camera (411) in real time, resets the destination in real time, and controls the robot (100a) to drive straight.

[0255] Meanwhile, the second destination (910) can be set to a location that satisfies the first and second conditions.

[0256] The first condition for setting the second destination (910) is that the direction of the second destination (910) must be facing straight ahead, perpendicular to the corner line of the table (810). Furthermore, the two-diff wheel method requires a certain distance to move left and right. Therefore, in order to approach a nearby destination in the correct direction, it is desirable to start moving with a certain degree of directional alignment with the destination from the starting point.

[0257] As a second condition for setting the second destination (910), the second destination (910) must be located within a short distance that is farther than the blind area that the depth camera (411) cannot see, and within a distance where the error does not exceed a standard value (e.g., 5 cm). For example, the second destination (910) may be located between the min distance and the max distance.

[0258] min distance: depth blind area 0.3m + slam position error 0.2m = Approach at least 0.5m away from the destination

[0259] Max distance: As the measurement distance increases, the measurement error of depth data increases and noise is added, causing the corners of the table to disappear and be detected. Accuracy decreases when the distance is more than 1m.

[0260] Figures 11 to 16 illustrate an embodiment of setting a second destination (910) between a min distance and a max distance based on the performance of a depth camera (411) and performing an approach drive.

[0261] Referring to Fig. 11, the original first destination (920) is set in a direction (921) perpendicular to the corner line of the table (810). The processor (180) can set a second destination (910), which is a virtual arrival point, at a distance of a reference distance d3 (0.5 to 1.0 m) in the opposite direction by referring to the direction of the angle information set in the first destination (920).

[0262] Additionally, the direction (1110) of the second destination (910) can be set in a direction perpendicular to the corner line of the table (810) in the same manner as the direction (921) of the first destination (920).

[0263] Meanwhile, referring to FIG. 12, when the robot (100a) arrives at the second destination (910), if a slam position error occurs, the direction (1210) may not match the set direction (1110) (S740). When the robot (100a) arrives at the second destination (910) in the normal driving mode, a slam position error may occur, and the direction (1210) may not exactly match the set direction (1110).

[0264] In this case, the direction of the robot (100a) can be corrected once more at the second destination (910) so that it is perpendicular to the edge line of the table (810) (S760). The processor (180) can rotate the robot (100a) so that the edge line of the table (810) and the robot (100a) are perpendicular to each other based on the shape of the table (810) detected by the depth camera (411).

[0265] Referring to Fig. 13, the direction (1310) of the robot corrected by rotation matches the set direction (1110).

[0266] Thereafter, the processor (180) can control the robot (100a) to drive straight in a table approach driving mode in the section between the second destination (910) and the table (810).

[0267] More preferably, the processor (180) can set the current position of the robot (100a) before starting proximity driving as the odometry origin.

[0268] The processor (180) can update the position of the table (810) visible to the depth camera (411) in real time to reset the destination in real time. In addition, the processor (180) can use an odometry-based position recognition method. The odometry-based position recognition method is difficult to use alone when driving long distances because position accumulation occurs. However, when moving short distances and approaching while updating the real-time destination, the origin of the odometry is reset based on the starting point, so that the slam position occurring during the entire driving can be excluded.

[0269] Referring to FIG. 14, the processor (180) detects the edge of the table (810) with the depth camera (411) and changes the destination in real time, and allows the robot (100a) to approach the table edge line at a right angle only up to the blind area distance (D4) (ex. 0.3 m) of the depth camera (411).

[0270] When approaching the table (810) within the blind area distance (D4) of the depth camera (411), the sensing accuracy of the table (810) decreases. Therefore, the processor (180) can control the robot (100a) to drive straight for an additional distance (D5) (e.g., 0.1 to 0.15 m) set based on odometry, rather than detecting and approaching the table (810) with the depth camera (411) from the blind area distance (D4). Accordingly, the robot (100a) can always be within 0.2 m from the table (810) and arrive at the corner in a direction perpendicular to the edge.

[0271] According to an embodiment, the processor (180) can move 0.15 m based on the table residual image remaining in the local map without destination correction according to the blind area of ​​the depth camera (411).

[0272] Referring to FIG. 16, the processor (180) additionally sets a virtual second destination (910) in the vertical direction, taking into account the arrival angle of the original destination. In addition, the processor (180) can control the robot (100a) to move toward the table (810) when it arrives near the virtual second destination (910). The processor (180) processes the arrival completion after the robot (100a) approaches the table (810) as much as possible within a collision-avoidable range.

[0273] As shown in Fig. 16, the robot (100a) moves from a virtual arrival location (910) toward an actual arrival location (910). The processor (180) monitors the sensor data of the sensing unit (140) and controls the robot (100a) to adhere to the table (810) as much as possible before stopping, thereby enabling the robot (100a) to remain constantly attached to the table (810) and stop.

[0274] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. An input interface for receiving a command to move to a first destination corresponding to a specified object; and A processor for setting a second destination spaced apart from the first destination; The above processor, Set the second destination so that the distance to the object is longer than the first destination, A robot that is controlled to drive close to the object after moving to the second destination.

2. In paragraph 1, further comprising a memory in which the map is stored; The above first destination is a destination fixedly stored on the map, The above second destination is a robot that is newly created when driving.

3. In paragraph 2, The above memory stores coordinate information and direction information of the first destination, The coordinates of the second destination are set to be a predetermined distance away from the coordinates of the first destination in the opposite direction of the direction in which the object is located. A robot in which the direction of the second destination is set in the vertical direction of the edge line of the object.

4. In paragraph 1, Including a depth camera; The second destination is a robot set at a position and direction where the depth camera can see the edge of the object.

5. In paragraph 4, The above processor, A robot that updates the position of an object seen by the depth camera during the above proximity driving, resets the second destination, and controls the robot to drive straight in the direction of the object.

6. In paragraph 4, The above second destination is, A robot determined based on the sensing area range of the above depth camera.

7. In paragraph 4, Further comprising driving wheels including left and rear wheels; The second destination is a robot set to a position and direction in which the object can be approached by driving by rotation of the driving wheel.

8. In paragraph 1, Further comprising an obstacle detection sensor that detects the presence of an obstacle and the distance to the obstacle; The above processor, A robot that is controlled to approach the object at the shortest distance without risk of collision based on sensing data from the obstacle detection sensor.

9. In paragraph 1, The above proximity driving is a driving of a robot that approaches the object while moving straight from the second destination toward the object.

10. In paragraph 1, The above specified object is a robot that is a table.

11. In paragraph 1, The above processor, Drive in the first driving mode to the above second destination, A robot that, upon arriving at the second destination, controls the robot to approach the object in a second driving mode.

12. In paragraph 11, The above first driving mode is a robot driving mode that uses position recognition based on SLAM (Simultaneous localization and mapping).

13. In paragraph 11, The above second driving mode is a robot driving mode that uses position recognition based on odometry information.

14. In paragraph 13, further comprising a wheel sensor for detecting the rotational speed of the driving wheel; The above odometry information is: A robot including sensing data of the above wheel sensor.

15. In paragraph 14, further comprising a lidar sensor; The above odometry information is: A robot based on sensing data from the above wheel sensor and the above lidar sensor.

16. In paragraph 12, The above processor, A robot that resets the position before the start of the proximity driving to the reference origin coordinates of the odometry information and controls the robot to start the proximity driving.

17. In paragraph 1, The above processor, After moving to the above second destination, if the direction of the robot does not match the vertical direction of the edge line of the object, A robot that rotates the robot so that the direction of the robot matches the vertical direction of the edge line of the object.

Images