Driving robot and control method thereof

Abstract

The present disclosure relates to a driving robot which can automatically change from a driving path-based driving method to an intuitive artificial intelligence-based driving method according to driving environment, and a control method thereof, wherein the driving robot includes: a sensor unit which acquires surrounding environment information; an input unit into which map data is inputted; a driving motor unit which moves the robot; and a control unit which determines the driving method on the basis of the surrounding environment information and map data to control the driving motor unit, wherein the control unit may: identify the location of the robot on the basis of the surrounding environment information and map data; determine the driving method on the basis of whether the location of the robot is identified; input direction information of a destination, distance information from the destination, and the surrounding environment information into a pre-trained neural network model to determine the optimal driving direction if the determined driving method is an artificial intelligence-based driving method on the basis of failing to identify the location of the robot; and control the driving motor unit so that the robot moves in the determined driving direction.

Description

Driving robot and its control method

[0001] The present disclosure relates to a driving robot capable of automatically changing from a driving path-based driving mode to an intuitive artificial intelligence-based driving mode depending on the driving environment, and a control method thereof.

[0002] In general, a robot is a machine that automatically processes or operates a given task with its own abilities, and the application fields of robots are generally classified into various fields such as industrial, medical, space, and underwater.

[0003] Recently, there has been an increasing trend of robots that can communicate or interact with humans through voice or gestures.

[0004] These robots can include various types of robots, such as guide robots that are placed in specific locations and provide various information to users, or home robots that are installed in homes.

[0005] In order for a robot to drive, it must find its location on a map, have a path to the destination location on the map, and be able to control its motors to follow that path.

[0006] The location of the robot on the map can be determined through the SLAM (Simultaneous Localization and Mapping) function, which compares the robot's sensor information, including camera images, lidar, and distances to obstacles around the robot, with map information.

[0007] In addition, the path generation to the destination is divided into a process of generating a global path that generates a path from the robot location to the destination location on the map and a local path to avoid obstacles around the robot, and the robot can control the motor through a motion controller to follow the local path that is ultimately generated.

[0008] However, the robot had a problem in that it could not know the path to the destination if it failed to recognize its own location or failed to generate a path such as a global path or a local path, and thus stopped for safety reasons until it received assistance from an outsider.

[0009] In this case, there was the inconvenience of having to indefinitely suspend services such as robot delivery and guidance until outside assistance was available, or having to move the robot to another location and restart it to re-recognize its location.

[0010] Therefore, in the future, there is a need to develop a driving robot that can drive autonomously using an intuitive artificial intelligence method even in driving environments such as robot location recognition failure and path generation failure.

[0011] The present disclosure aims to solve the above-mentioned problems and other problems.

[0012] The present disclosure provides a driving robot and a control method thereof that can stably drive to a destination and complete service performance even in a poor surrounding environment by automatically changing from a driving path-based driving mode to an artificial intelligence-based driving mode to determine an optimal driving direction when robot location recognition or path generation fails.

[0013] According to one embodiment of the present disclosure, a driving robot includes a sensor unit for acquiring information about a surrounding environment, an input unit for inputting map data, a drive motor unit for moving the robot, and a control unit for determining a driving method based on the surrounding environment information and the map data and controlling the drive motor unit, wherein the control unit confirms the location of the robot based on the surrounding environment information and the map data, determines the driving method based on whether the location of the robot has been confirmed, and if the determined driving method is an artificial intelligence-based driving method based on a failure to confirm the location of the robot, inputs information about the direction of the destination, information about the distance from the destination, and information about the surrounding environment into a pre-learned neural network model to determine an optimal driving direction, and controls the drive motor unit so that the robot moves in the determined driving direction.

[0014] A control method of a driving robot according to one embodiment of the present disclosure is a control method of a driving robot including a drive motor unit, the control method including a step of acquiring surrounding environment information and map data, a step of confirming a robot position based on the surrounding environment information and the map data, a step of determining a driving method depending on whether the robot position has been confirmed, a step of inputting destination direction information, distance information from the destination, and surrounding environment information into a pre-learned neural network model to determine an optimal driving direction if the determined driving method is an artificial intelligence-based driving method depending on failure to confirm the robot position, and a step of controlling the drive motor unit so that the driving robot moves in the determined driving direction.

[0015] According to one embodiment of the present disclosure, if the robot location recognition or path generation fails, the driving robot automatically changes from a path-based driving mode to an artificial intelligence-based driving mode to determine an optimal driving direction, thereby stably driving to a destination and completing service performance even in a poor surrounding environment.

[0016] In this way, the present disclosure automatically changes the driving method based on the driving path to an intuitive artificial intelligence-based driving method, so that the robot can drive stably and continuously to the destination even when the robot loses the path to the destination or is unable to drive.

[0017] According to the present disclosure, when a robot fails to recognize its own location or fails to generate a path to a destination, it does not stop or perform a search operation to re-recognize its location, so that services (delivery, guidance, etc.) provided by the robot can be provided without interruption to the destination.

[0018] The present disclosure provides a method for a robot to recognize its surroundings and move toward the destination by avoiding obstacles using an intuitive artificial intelligence method, without needing a map or driving path, if it only knows the direction to the destination, even if the robot fails to recognize its own location or fails to generate a path to the destination.

[0019] FIG. 1 is a drawing for explaining a driving robot according to one embodiment of the present disclosure.

[0020] FIG. 2 is a drawing for explaining a control unit that performs a driving path-based driving method of a driving robot according to one embodiment of the present disclosure.

[0021] FIG. 3 is a drawing for explaining a control unit that performs an artificial intelligence-based driving method of a driving robot according to one embodiment of the present disclosure.

[0022] FIG. 4 and FIG. 5 are drawings for explaining conditions for determining the driving method of a driving robot according to one embodiment of the present disclosure.

[0023] FIG. 6 is a drawing for explaining a hybrid driving method of a driving robot according to one embodiment of the present disclosure.

[0024] FIGS. 7 to 10 are diagrams for explaining a neural network model learning process of a driving robot according to an embodiment of the present disclosure.

[0025] FIGS. 11 to 13 are drawings for explaining corridor driving of a driving robot according to an embodiment of the present disclosure.

[0026] FIG. 14 is a flowchart for explaining a control process of a driving robot according to an embodiment of the present disclosure.

[0027] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present disclosure.

[0028] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0029] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0030] Additionally, throughout this specification, the terms "neural network," "neural network," and "network function" may be used interchangeably. A neural network may be comprised of a set of interconnected computational units, which may generally be referred to as "nodes." These "nodes" may also be referred to as "neurons." A neural network comprises at least two or more nodes. The nodes (or neurons) constituting the neural networks may be interconnected by one or more "links."

[0031] Artificial Intelligence (AI) refers to the field of study that studies artificial intelligence or the methodologies for creating it, while Machine Learning (ML) refers to the field that defines various problems in the field of AI and studies the methodologies for solving them. Machine learning is also defined as an algorithm that improves performance on a task through consistent experience.

[0032] An artificial neural network (ANN) is a model used in machine learning. It can refer to a model with problem-solving capabilities, comprised of artificial neurons (nodes) formed by the connection of synapses. An ANN can be defined by the connection patterns between neurons in different layers, the learning process that updates model parameters, and the activation function that generates output values.

[0033] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer may contain one or more neurons, and the artificial neural network may include synapses connecting neurons. In an artificial neural network, each neuron can output a function value of an activation function based on input signals, weights, and biases received through the synapses.

[0034] Model parameters are parameters determined through learning, including synaptic connection weights and neuron biases. Hyperparameters are parameters that must be set before learning in machine learning algorithms, including the learning rate, number of iterations, mini-batch size, and initialization function.

[0035] The goal of artificial neural network training can be seen as determining model parameters that minimize a loss function. The loss function can be used as an indicator for determining optimal model parameters during the artificial neural network training process.

[0036] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

[0037] Supervised learning refers to a method for training an artificial neural network given labels for training data. Labels can refer to the correct answer (or output) that the artificial neural network must infer when the training data is input to the artificial neural network. Unsupervised learning refers to a method for training an artificial neural network without given labels for the training data. Reinforcement learning refers to a learning method in which an agent defined within a given environment is trained to select actions or action sequences that maximize cumulative rewards in each state.

[0038] Among artificial neural networks, machine learning implemented with a deep neural network (DNN) containing multiple hidden layers is sometimes called deep learning, and deep learning is a subset of machine learning. Hereinafter, "machine learning" is used to encompass deep learning.

[0039] FIG. 1 is a drawing for explaining a driving robot according to one embodiment of the present disclosure.

[0040] As illustrated in FIG. 1, the driving robot (10) of the present disclosure may include a sensor unit (100) that acquires surrounding environment information, an input unit (200) that inputs map data, a drive motor unit (300) that moves the robot, and a control unit (400) that determines a driving method based on the surrounding environment information and map data and controls the drive motor unit (300).

[0041] Here, the sensor unit (100) may include at least one of a 3D camera, a lidar sensor, an ultrasonic sensor, and a distance sensor.

[0042] In addition, the control unit (400) determines the robot's location based on the surrounding environment information and map data, determines the driving method based on whether the robot's location has been confirmed, and if the determined driving method is an artificial intelligence-based driving method based on failure to confirm the robot's location, the control unit (400) can input the destination's direction information, the distance information from the destination, and the surrounding environment information into a pre-learned neural network model to determine the optimal driving direction, and control the drive motor unit (300) so that the robot moves in the determined driving direction.

[0043] When confirming the location of the robot, the control unit (400) can obtain surrounding environment information and map data, and match the map data with the surrounding environment information to confirm the current location of the robot.

[0044] And, when determining the driving method, the control unit (400) can determine the first driving method based on the driving path if the robot position is confirmed successfully, and can determine the second driving method based on artificial intelligence if the robot position is confirmed unsuccessfully.

[0045] Here, the control unit (400) determines the current driving mode if the current driving mode is determined as the first driving mode based on the driving path, and if the current driving mode is the second driving mode based on artificial intelligence, it changes to the first driving mode that determined the second driving mode, and if the current driving mode is the first driving mode based on the driving path, it can maintain the first driving mode.

[0046] The control unit (400) determines the current driving mode when it is determined as the artificial intelligence-based second driving mode, and if the current driving mode is the driving path-based first driving mode, it changes to the second driving mode that determined the first driving mode, and if the current driving mode is the artificial intelligence-based second driving mode, it can maintain the second driving mode.

[0047] Next, if the determined driving method is a driving method based on a driving path based on successful confirmation of the robot's location, the control unit (400) can generate a driving path based on robot location information, map data, destination location information, and surrounding environment information, and control the drive motor unit (300) to cause the robot to move along the generated driving path.

[0048] Here, the control unit (400) can change the driving path-based driving method to an artificial intelligence-based driving method when generating a driving path if the generation of the driving path fails.

[0049] In addition, the control unit (400) can generate a global driving path based on robot location information, map data, and destination location information when generating a driving path, generate a local driving path based on the global driving path and surrounding environment information, and control the drive motor unit (300) so that the robot moves along the generated local driving path.

[0050] Here, when generating a global driving path, the control unit (400) can change the driving path-based driving method to an artificial intelligence-based driving method if the generation of the global driving path fails.

[0051] In addition, when generating a local driving path, the control unit (400) can change the driving path-based driving method to an artificial intelligence-based driving method if the generation of the local driving path fails.

[0052] Next, the control unit (400) determines the first driving direction based on the direction information of the destination and the distance information from the destination when determining the optimal driving direction, checks whether there is an obstacle located in the first driving direction based on the surrounding environment information, and if there is an obstacle in the first driving direction, changes the first driving direction to the second driving direction.

[0053] Here, when checking whether an obstacle exists in the first driving direction, the control unit (400) can determine the first driving direction as the optimal driving direction if there is no obstacle in the first driving direction.

[0054] In addition, when checking whether an obstacle exists in the first driving direction, the control unit (400) can check whether the obstacle is a corridor wall if there is an obstacle in the first driving direction, and if the obstacle is a corridor wall, recognize the direction along the corridor wall as the destination direction.

[0055] Here, when checking whether the obstacle is a corridor wall, the control unit (400) can recognize the detected straight line as an obstacle if a straight line is detected in the first driving direction based on the surrounding environment information, check whether the detected straight line is longer than a set length, and recognize the obstacle as a corridor wall if the detected straight line is longer than the set length.

[0056] Next, the neural network model can be pre-trained using a reinforcement learning method that determines the driving direction based on the direction information of the destination, the distance information from the destination, and the surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

[0057] Here, the neural network model can be pre-trained using reinforcement learning, which provides a larger reward value as the arrival time to the destination is shortened when determining the driving direction.

[0058] In some cases, the neural network model may be pre-trained using a reinforcement learning method in which it receives additional noise information from a noise generator, determines a driving direction based on the noise information, destination direction information, distance information from the destination, and surrounding environment information, and provides a reward or penalty value based on the determined driving direction.

[0059] Here, the neural network model can be pre-trained using reinforcement learning, which provides a reward value according to a decrease in the arrival time to the destination when determining the driving direction, and a penalty value according to the occurrence of a collision, proximity to an obstacle, and an increase in the arrival time to the destination.

[0060] For example, noise information may include noise corresponding to sensor input errors of the robot and motion errors of the robot.

[0061] In another case, the neural network model can be pre-trained using a reinforcement learning method in which it additionally receives a direction indicator, determines a driving direction based on the direction indicator, information about the direction of the destination, information about the distance to the destination, and information about the surrounding environment, and provides a reward or penalty value according to the determined driving direction.

[0062] For example, the direction indicators may include a straight direction indicator, a right turn direction indicator, and a left turn direction indicator.

[0063] In another case, the neural network model can be pre-trained using a reinforcement learning method in which, when inputting surrounding environment information, the surrounding environment information is preprocessed to be inverted left and right, the inverted surrounding environment information is periodically learned, the driving direction is determined based on the inverted surrounding environment information, the direction information of the destination, the distance information to the destination, and the surrounding environment information, and a reward value or penalty value is provided according to the determined driving direction.

[0064] In another case, the neural network model can be pre-trained using a reinforcement learning method that additionally receives noise information from a noise generator, additionally receives a direction indicator, determines a driving direction based on the noise information, the direction indicator, direction information of the destination, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

[0065] In another case, the neural network model can be pre-trained using a reinforcement learning method in which, when surrounding environment information is input, the surrounding environment information is preprocessed to be inverted left and right, the inverted surrounding environment information is periodically learned, additional noise information is input from a noise generator, an additional direction indicator is input, and the driving direction is determined based on the inverted surrounding environment information, the noise information, the direction indicator, the direction information of the destination, the distance information from the destination, and the surrounding environment information, and a reward value or penalty value is provided according to the determined driving direction.

[0066] In addition, the control unit (400) may include a noise generator that generates noise information, and when noise information is obtained from the noise generator, the neural network model may be subjected to reinforcement learning to determine a driving direction by inputting the noise information, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information into the neural network model.

[0067] Here, the noise generator can add noise to the robot's surrounding sensor information and the robot's linear speed and rotational speed to generate noise corresponding to the robot's sensor input error and the robot's motion error.

[0068] In addition, the control unit (400) may include a driving direction indicator generator that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning of a neural network model for turns. When a driving direction indicator is obtained from the driving direction indicator generator, the driving direction indicator, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information may be input into the neural network model to perform reinforcement learning to determine the driving direction.

[0069] In addition, the control unit (400) may include a replay buffer unit that collects and stores peripheral sensor information of the robot used for learning the neural network model, a preprocessing unit that extracts peripheral sensor information from the replay buffer unit and preprocesses the peripheral sensor information so that the peripheral sensor information is inverted left and right, and a learning unit that inputs the inverted peripheral sensor information into the neural network model for pre-learning, and when the learning result value is obtained by the learning unit, the learning result value information corresponding to the inverted peripheral sensor information, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and peripheral environment information may be input into the neural network model to perform reinforcement learning so that the neural network model determines the driving direction.

[0070] In addition, the control unit (400) may include a noise generator that generates noise information, and a driving direction indicator generator that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning a neural network model for turns. When noise information is obtained from the noise generator and a driving direction indicator is obtained from the driving direction indicator generator, the noise information, the driving direction indicator, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information are input into the neural network model to perform reinforcement learning so that the neural network model determines the driving direction.

[0071] In addition, the control unit (400) may include a noise generator that generates noise information, a driving direction indicator generator that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning a neural network model for turns, a replay buffer unit that collects and stores peripheral sensor information of the robot used for learning the neural network model, a preprocessing unit that extracts peripheral sensor information from the replay buffer unit and preprocesses the peripheral sensor information so that the peripheral sensor information is inverted left and right, and a learning unit that inputs the inverted peripheral sensor information into a neural network model for pre-learning, wherein the noise information is obtained from the noise generator, the driving direction indicator is obtained from the driving direction indicator generator, and when the learning result value is obtained by the learning unit, the learning result value information corresponding to the noise information, the driving direction indicator, the left and right inverted peripheral sensor information, result value information including a reward or penalty according to reinforcement learning, information on the direction of the destination, information on the distance to the destination, and information on the surrounding environment are input into the neural network model to perform reinforcement learning so that the neural network model determines the driving direction.

[0072] In this way, the driving robot of the present disclosure can complete service performance by driving stably to a destination even in a poor surrounding environment by automatically changing from a driving path-based driving mode to an artificial intelligence-based driving mode to determine an optimal driving direction when robot position recognition or path generation fails.

[0073] The present disclosure enables a robot to drive steadily and continuously to a destination by automatically changing from a path-based driving method to an intuitive artificial intelligence-based driving method even when the robot loses the path to the destination or is unable to drive.

[0074] According to the present disclosure, when a robot fails to recognize its own location or fails to generate a path to a destination, it does not stop or perform a search operation to re-recognize its location, so that services (delivery, guidance, etc.) provided by the robot can be provided without interruption to the destination.

[0075] The present disclosure provides a method for a robot to recognize its surroundings and move toward the destination by avoiding obstacles using an intuitive artificial intelligence method, without needing a map or driving path, if it only knows the direction to the destination, even if the robot fails to recognize its own location or fails to generate a path to the destination.

[0076] FIG. 2 is a drawing for explaining a control unit that performs a driving path-based driving method of a driving robot according to one embodiment of the present disclosure.

[0077] As illustrated in FIG. 2, the control unit (400) of the present disclosure can determine the current location of the robot based on the surrounding environment information and map data, and if the current location of the robot is confirmed successfully, determine the driving method based on the driving path, and if the current location of the robot is confirmed unsuccessfully, determine the driving method based on artificial intelligence.

[0078] As shown in Fig. 2, if the determined driving method is a driving method based on a driving path that is determined by successful confirmation of the robot's location, the control unit (400) can generate a driving path based on robot location information, map data, destination location information, and surrounding environment information, and control the robot to move along the generated driving path.

[0079] Here, the control unit (400) may include a SLAM (Simultaneous Localization and Mapping) (412), a global path planner (414), a local path planner (416), and a motion controller.

[0080] First, the SLAM (412) of the control unit (400) can receive map information and information surrounding the robot for matching with the map and compare this information to recognize the current location of the robot.

[0081] That is, SLAM (412) can determine the location of the robot by comparing the robot's sensor information, including camera images, lidar, distances to obstacles around the robot, and map information.

[0082] Next, the global path planner (414) of the control unit (400) can receive map information, the location of the destination, and the location of the robot from SLAM (412) and generate a global path including a path from the location of the robot to the location of the destination on the map.

[0083] Next, the local path planner (416) of the control unit (400) can receive a global path from the global path planner (414) and input obstacle information around the robot for obstacle avoidance from a sensor to generate a local path for avoiding obstacles around the robot.

[0084] And, the motion controller (418) of the control unit (400) can control the movement of the robot by controlling the motor of the robot to follow the local path.

[0085] FIG. 3 is a drawing for explaining a control unit that performs an artificial intelligence-based driving method of a driving robot according to one embodiment of the present disclosure.

[0086] As illustrated in FIG. 3, the control unit (400) of the present disclosure can determine the current location of the robot based on the surrounding environment information and map data, and if the current location of the robot is confirmed successfully, determine the driving method based on the driving path, and if the current location of the robot is confirmed unsuccessfully, determine the driving method based on artificial intelligence.

[0087] As shown in Fig. 3, if the determined driving method is an artificial intelligence-based driving method due to a failure to confirm the robot's position, the control unit (400) inputs the direction information of the destination, the distance information to the destination, and the surrounding environment information into a pre-learned neural network model (420) to determine the optimal driving direction, and controls the robot to move in the determined driving direction.

[0088] Here, the neural network model (420) of the motion controller can be pre-trained using a reinforcement learning method that determines a driving direction based on destination direction information, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

[0089] Here, the neural network model (420) can be pre-trained using reinforcement learning, which provides a greater reward value as the arrival time to the destination is shortened when determining the driving direction.

[0090] For example, the learning method of reinforcement learning is as follows.

[0091] Reinforcement learning is a method in which an agent (robot, in this case) takes actions (actions, in this case robot motion control) based on observations (observations, information measured from sensors) in the surrounding environment, and receives rewards (rewards, if the action produces a good result, a reward, if it produces a bad result, a penalty; that is, if the robot avoids an obstacle and moves closer to the target point, a + score is given, and if the robot collides with an obstacle or moves away from an obstacle, a score is given) based on the results, and the agent (robot) learns to take actions (robot motion control) on its own to receive better rewards based on the environment (sensor information).

[0092] When training with reinforcement learning using an actual robot, training can be conducted primarily in a simulation environment, as incorrect movements of the robot can lead to accidents by colliding with obstacles.

[0093] In a simulation environment, a reinforcement learning motion controller can control a simulation robot to perform various actions.

[0094] Also, if the robot's movement results are good (it reaches the destination well or avoids obstacles well), a reward value can be given, and if the robot's movement results are bad (it hits an obstacle or moves around too far), a penalty value can be given so that the reinforcement learning motion controller learns in the direction of receiving the best reward value.

[0095] The learning of the reinforcement learning robot motion controller is designed so that the robot receives a higher reward value the faster it avoids obstacles and reaches the destination, by learning in a direction that receives a good reward in a simulation environment.

[0096] In this way, the present disclosure can have the following strengths by utilizing a reinforcement learning method for robot driving.

[0097] The present disclosure has a simple system structure and can reduce the processing load compared to existing methods because it drives without path generation through an artificial intelligence robot driving controller using reinforcement learning.

[0098] In addition, the present disclosure uses an artificial intelligence controller that utilizes reinforcement learning to intuitively avoid obstacles and drive like a human, as long as empty spaces between obstacles are visible and the destination direction is known, so the robot control speed is faster than that of existing controllers and driving motions similar to those of a human are possible.

[0099] Additionally, reinforcement learning methods can enable learning to behave differently depending on the learning intention.

[0100] For example, the present disclosure can enable learning for a robot to operate in different forms according to its purpose, such as a first learning method that follows a path well if there is a path, a second learning method that ignores the path and only moves well toward the destination, a third learning method that avoids obstacles well if there are many obstacles, and a fourth learning method that allows the robot to hit obstacles to some extent but unconditionally reaches the destination quickly.

[0101] In addition, the present disclosure can train a neural network model of a motion controller using a learning method that allows the motion controller to drive in a direction by only inputting the direction and distance to the destination without a path, and a learning method that avoids obstacles when encountering them while driving to avoid collisions with the obstacles.

[0102] FIG. 4 and FIG. 5 are drawings for explaining conditions for determining the driving method of a driving robot according to one embodiment of the present disclosure.

[0103] As illustrated in FIGS. 4 and 5, the present disclosure determines the robot position based on surrounding environment information and map data, determines a driving method based on whether the robot position has been confirmed, and if the determined driving method is an artificial intelligence-based driving method based on failure to confirm the robot position, inputs destination direction information, distance information from the destination, and surrounding environment information into a pre-learned neural network model to determine an optimal driving direction, and controls a drive motor so that the robot moves in the determined driving direction.

[0104] The present disclosure may have a hybrid switching module that can determine a first driving method based on a driving path when confirmation of a robot position is successful, and can determine a second driving method based on artificial intelligence when confirmation of a robot position fails when determining a driving method.

[0105] As shown in Fig. 4, in the present disclosure, if the robot location recognition by SLAM (412) is successful and the path generation such as the global path and local path by GPP (414) and LPP (416) is successful, the first driving method based on the driving path can be determined through the hybrid switching module.

[0106] On the other hand, in the present disclosure, when robot position recognition by SLAM (412) fails and path generation such as global path and local path by GPP (414) and LPP (416) fails, a second driving method based on intuitive artificial intelligence can be determined through a hybrid switching module.

[0107] As shown in FIG. 5, in the present disclosure, if the robot location recognition by SLAM (412) is successful and the path generation such as the global path and local path by GPP (414) and LPP (416) is successful, the shape of the wall on the map and the obstacle input from the robot sensor appear similar based on the robot location, so the robot can be controlled based on the driving path since there is no problem with the location of the robot on the map.

[0108] On the other hand, in the present disclosure, when the robot location recognition by SLAM (412) fails and the path generation such as the global path and local path by GPP (414) and LPP (416) fails, the shape of the wall on the map and the obstacle input from the robot sensor appear different based on the robot location, so the robot can be controlled based on intuitive artificial intelligence because the robot failed to recognize the location on the map.

[0109] FIG. 6 is a drawing for explaining a hybrid driving method of a driving robot according to one embodiment of the present disclosure.

[0110] As illustrated in FIG. 6, the present disclosure determines the current driving mode when the first driving mode based on the driving path is determined, and if the current driving mode is an artificial intelligence-based second driving mode, the second driving mode is changed to the first driving mode determined, and if the current driving mode is a first driving mode based on the driving path, the first driving mode can be maintained.

[0111] In addition, the present disclosure determines the current driving mode when the second driving mode based on artificial intelligence is determined, and if the current driving mode is a first driving mode based on a driving path, the first driving mode is changed to the second driving mode determined, and if the current driving mode is an artificial intelligence-based second driving mode, the second driving mode can be maintained.

[0112] In this way, the present disclosure can generate a driving path based on robot location information, map data, destination location information, and surrounding environment information if the determined driving method is a driving path-based driving method based on successful confirmation of the robot location, and can control the drive motor so that the robot moves along the generated driving path.

[0113] Here, the present disclosure can change the driving path-based driving method to an artificial intelligence-based driving method when generating a driving path, if the generation of the driving path fails.

[0114] FIGS. 7 to 10 are diagrams for explaining a neural network model learning process of a driving robot according to an embodiment of the present disclosure.

[0115] As illustrated in FIG. 7, the present disclosure, in the case of an artificial intelligence-based driving method due to a failure to confirm the robot's position, can input destination direction information, distance information from the destination, and surrounding environment information into a pre-learned neural network model (600) to determine an optimal driving direction.

[0116] Here, the neural network model (600) can be pre-trained using a reinforcement learning method that determines a driving direction based on direction information of the destination, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

[0117] That is, the neural network model (600) can be pre-trained using reinforcement learning, which provides a greater reward value as the arrival time to the destination is shortened when determining the driving direction.

[0118] The present disclosure can be implemented as a simulation environment (500) for reinforcement learning so as to reflect actual delivery and driving sites as much as possible.

[0119] For example, as shown in Fig. 7, as a simulation environment (500) for reinforcement learning, a delivery environment (510) crowded with actual people can be simulated and used for learning, a situation (520) of avoiding obstacles in a hallway can be simulated and used for learning, or other various obstacle avoidance situations (530) can be simulated and used for learning.

[0120] As illustrated in FIG. 8, the present disclosure may include a noise generator (710) that generates noise information. When noise information is obtained from the noise generator (710), the noise information, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information are input into the neural network model (600) so that the neural network model (600) can be subjected to reinforcement learning to determine a driving direction.

[0121] Here, the noise generator (710) can add noise to the robot's surrounding sensor information and the robot's linear speed and rotational speed to generate noise corresponding to the robot's sensor input error and the robot's motion error.

[0122] As shown in Fig. 8, the neural network model (600) may be pre-trained using a reinforcement learning method that additionally receives noise information from a noise generator (710), determines a driving direction based on the noise information, direction information of the destination, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

[0123] Here, the neural network model (600) can be pre-trained using reinforcement learning to provide a reward value according to a decrease in the arrival time to the destination when determining the driving direction, and a penalty value according to the occurrence of a collision, proximity to an obstacle, and an increase in the arrival time to the destination.

[0124] For example, noise information may include noise corresponding to sensor input errors of the robot and motion errors of the robot.

[0125] In addition, the present disclosure can be used for learning by simulating a delivery environment (510) crowded with actual people as a simulation environment (500) for reinforcement learning, a situation (520) of avoiding obstacles in a hallway while delivering, or other various obstacle avoidance situations (530) by simulating them.

[0126] As illustrated in FIG. 9, the present disclosure may include a noise generator (710) that generates noise information, and a driving direction indicator generator (720) that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning a neural network model for turns.

[0127] Here, the present disclosure obtains noise information from a noise generator (710) and obtains a driving direction indicator from a driving direction indicator generator (720), and inputs the noise information, the driving direction indicator, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information into the neural network model (600) to enable reinforcement learning to determine the driving direction.

[0128] That is, the neural network model (600) may be pre-trained using a reinforcement learning method that additionally receives noise information from a noise generator (710), additionally receives a direction indicator from a direction indicator generator (720), determines a direction based on the noise information, the direction indicator, direction information of the destination, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined direction.

[0129] For example, the direction indicators may include a straight direction indicator, a right turn direction indicator, and a left turn direction indicator.

[0130] In addition, the present disclosure can be used for learning by simulating a delivery environment (510) crowded with actual people as a simulation environment (500) for reinforcement learning, a situation (520) of avoiding obstacles in a hallway while delivering, or other various obstacle avoidance situations (530) by simulating them.

[0131] As illustrated in FIG. 10, the present disclosure may include a noise generator (710) for generating noise information, a driving direction indicator generator (720) for generating driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning a neural network model for turns, a replay buffer unit (730) for collecting and storing peripheral sensor information of a robot used for learning a neural network model (600), a preprocessing unit (740) for extracting peripheral sensor information from the replay buffer unit (730) and preprocessing the peripheral sensor information so that it is inverted left and right, and a learning unit for inputting the inverted peripheral sensor information into the neural network model (600) for pre-learning.

[0132] The present disclosure obtains noise information from a noise generator (710), obtains a driving direction indicator from a driving direction indicator generator (720), and obtains a learning result value with a learning unit. Then, the learning result value information corresponding to the noise information, the driving direction indicator, the left-right inverted surrounding sensor information, the result value information including a reward or penalty according to reinforcement learning, the direction information of the destination, the distance information from the destination, and the surrounding environment information are input into the neural network model (600) so that the neural network model (600) can be subjected to reinforcement learning to determine the driving direction.

[0133] In addition, the present disclosure can be used for learning by simulating a delivery environment (510) crowded with actual people as a simulation environment (500) for reinforcement learning, a situation (520) of avoiding obstacles in a hallway while delivering, or other various obstacle avoidance situations (530) by simulating them.

[0134] In some cases, the present disclosure may include a driving direction indicator generator that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning a neural network model for turns, and when a driving direction indicator is obtained from the driving direction indicator generator, the driving direction indicator, result value information including a reward or penalty according to reinforcement learning, direction information of a destination, distance information from the destination, and surrounding environment information may be input into the neural network model to perform reinforcement learning to determine a driving direction.

[0135] Here, the neural network model can be pre-trained using a reinforcement learning method that additionally receives a direction indicator, determines a driving direction based on the direction indicator, destination direction information, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

[0136] In another case, the present disclosure may include a replay buffer unit that collects and stores peripheral sensor information of a robot used for learning a neural network model, a preprocessing unit that extracts peripheral sensor information from the replay buffer unit and preprocesses the peripheral sensor information so that the peripheral sensor information is inverted left and right, and a learning unit that inputs the inverted peripheral sensor information into a neural network model for pre-training, and when a learning result value is obtained by the learning unit, the neural network model may be reinforced by inputting learning result value information corresponding to the inverted peripheral sensor information, result value information including a reward or penalty according to reinforcement learning, direction information of a destination, distance information from the destination, and peripheral environment information into the neural network model to determine a driving direction.

[0137] Here, the neural network model can be pre-trained using a reinforcement learning method in which, when surrounding environment information is input, the surrounding environment information is preprocessed to be inverted left and right, the left and right inverted surrounding environment information is periodically learned, and the driving direction is determined based on the left and right inverted surrounding environment information, the direction information of the destination, the distance information to the destination, and the surrounding environment information, and a reward value or penalty value is provided according to the determined driving direction.

[0138] In another case, the neural network model can be pre-trained using a reinforcement learning method in which, when surrounding environment information is input, the surrounding environment information is preprocessed to be inverted left and right, the inverted surrounding environment information is periodically learned, additional noise information is input from a noise generator, an additional direction indicator is input, and the driving direction is determined based on the inverted surrounding environment information, the noise information, the direction indicator, the direction information of the destination, the distance information from the destination, and the surrounding environment information, and a reward value or penalty value is provided according to the determined driving direction.

[0139] FIGS. 11 to 13 are drawings for explaining corridor driving of a driving robot according to an embodiment of the present disclosure.

[0140] As illustrated in FIGS. 11 to 13, the present disclosure determines a first driving direction based on direction information of a destination and distance information from the destination when determining an optimal driving direction, checks whether an obstacle is located in the first driving direction based on surrounding environment information, and if an obstacle is present in the first driving direction, changes the first driving direction to a second driving direction.

[0141] Here, the present disclosure can determine the first driving direction as the optimal driving direction when checking whether there is an obstacle located in the first driving direction and if there is no obstacle in the first driving direction.

[0142] The present disclosure, when checking whether there is an obstacle located in the driving direction, checks whether the obstacle is a corridor wall if there is an obstacle in the first driving direction, and if the obstacle is a corridor wall, recognizes the direction along the corridor wall as the direction of the destination.

[0143] Here, the present disclosure, when checking whether an obstacle is a corridor wall, if a straight line is detected in the first driving direction based on surrounding environment information, the detected straight line is recognized as an obstacle, and if it is checked whether the detected straight line is longer than a set length, the obstacle can be recognized as a corridor wall.

[0144] As shown in Fig. 11, the present disclosure can control the movement of the robot so that it can reach the destination at a high speed by avoiding only obstacles in front when the destination is at the end of the corridor.

[0145] However, as shown in Fig. 12, if the destination is located in another parallel corridor, the motion controller can control the robot to drive carefully while reducing its driving speed to avoid the obstacle, because it considers the corridor wall as an obstacle to be avoided.

[0146] That is, in a corridor driving situation of a robot, when driving with a reinforcement learning motion controller, if the destination is at the end of the corridor, the robot can reach the destination quickly, but if it is in a position that has turned at the intersection at the end of the corridor or is in another corridor in parallel, the corridor wall may be recognized as an obstacle that must be simply bypassed, and the robot's driving speed may slow down.

[0147] Accordingly, as shown in FIG. 13, the present disclosure can control the robot to change the direction of following the corridor wall toward the destination when the robot is in a corridor and the direction to the destination is not the same as the corridor direction or is not in the same corridor, and can control the robot to drive toward the destination when the destination and the corridor direction are the same or are in the same corridor.

[0148] That is, the present disclosure can control the robot with a reinforcement learning motion controller to move in the direction along the corridor wall as the destination direction instead of the actual destination direction when a corridor wall is detected from the information on obstacles around the robot detected by the robot's sensor.

[0149] The presence or absence of a corridor wall can be determined by using a method of detecting a straight line longer than a certain length from sensor information surrounding the robot.

[0150] Fixed obstacles and moving obstacles (such as people) may not be recognized as walls because their length is perceived as small.

[0151] FIG. 14 is a flowchart for explaining a control process of a driving robot according to an embodiment of the present disclosure.

[0152] As illustrated in FIG. 14, the present disclosure can acquire surrounding environment information and map data when the robot drives (S10), and confirm the robot location based on the surrounding environment information and map data (S20).

[0153] And, the present disclosure can perform hybrid switching that determines the driving method depending on whether the robot position is confirmed (S30).

[0154] Next, in the present disclosure, if the determined driving method is a driving method based on a driving path according to successful confirmation of the robot position, a driving path can be generated based on robot position information, map data, destination position information, and surrounding environment information (S40), and a robot control command can be generated so that the robot moves along the generated driving path (S50).

[0155] Here, the present disclosure can perform hybrid switching to change the driving path-based driving method to an artificial intelligence-based driving method if the generation of the driving path fails (S30).

[0156] Next, in the present disclosure, if the determined driving method is an artificial intelligence-based driving method due to a failure in confirming the robot's location, the direction information of the destination, the distance information from the destination, and the surrounding environment information are input into a pre-learned neural network model to determine an optimal driving direction, and a robot control command can be generated so that the driving robot moves in the determined driving direction (S60).

[0157] In this way, the present disclosure determines an optimal driving direction by automatically changing from a driving path-based driving method to an artificial intelligence-based driving method in the event of a robot location recognition failure or a path generation failure, thereby enabling stable driving to a destination and completing service performance even in a poor surrounding environment.

[0158] In this way, the present disclosure automatically changes the driving method based on the driving path to an intuitive artificial intelligence-based driving method, so that the robot can drive stably and continuously to the destination even when the robot loses the path to the destination or is unable to drive.

[0159] According to the present disclosure, when a robot fails to recognize its own location or fails to generate a path to a destination, it does not stop or perform a search operation to re-recognize its location, so that services (delivery, guidance, etc.) provided by the robot can be provided without interruption to the destination.

[0160] The present disclosure provides a method for a robot to recognize its surroundings and move toward the destination by avoiding obstacles using an intuitive artificial intelligence method, without needing a map or driving path, if it only knows the direction to the destination, even if the robot fails to recognize its own location or fails to generate a path to the destination.

[0161] According to the driving robot according to the present disclosure, in the event of a failure in robot position recognition or path generation, the driving method based on the driving path is automatically changed to an artificial intelligence-based driving method to determine an optimal driving direction, thereby stably driving to the destination and completing the service, and thus has significant industrial applicability.

Claims

1. Sensor unit that obtains information about the surrounding environment; Input section where map data is entered; A drive motor that moves the robot; and It includes a control unit that controls the driving motor unit by determining a driving method based on the above-mentioned surrounding environment information and map data, The above control unit, A driving robot characterized in that the robot location is confirmed based on the above-mentioned surrounding environment information and map data, a driving method is determined based on whether the robot location is confirmed, and if the determined driving method is an artificial intelligence-based driving method due to failure to confirm the robot location, the direction information of the destination, the distance information from the destination, and the surrounding environment information are input into a pre-learned neural network model to determine an optimal driving direction, and the driving motor unit is controlled so that the robot moves in the determined driving direction.

2. In paragraph 1, The above control unit, A driving robot characterized in that, when confirming the location of the robot, the surrounding environment information and map data are acquired, and the current location of the robot is confirmed by matching the map data and the surrounding environment information.

3. In paragraph 1, The above control unit, A driving robot characterized in that, when determining the driving method, if the robot position is confirmed successfully, the driving method is determined as a first driving method based on a driving path, and if the robot position is confirmed unsuccessfully, the driving method is determined as a second driving method based on artificial intelligence.

4. In paragraph 1, The above control unit, A driving robot characterized in that, if the driving method determined above is a driving method based on a driving path according to successful confirmation of the robot location, a driving path is generated based on the robot location information, the map data, the location information of the destination, and the surrounding environment information, and the driving motor unit is controlled so that the robot moves along the generated driving path.

5. In paragraph 1, The above control unit, A driving robot characterized in that, when determining the optimal driving direction, a first driving direction is determined based on direction information of the destination and distance information from the destination, the presence or absence of an obstacle located in the first driving direction is checked based on the surrounding environment information, and if an obstacle exists in the first driving direction, the first driving direction is changed to a second driving direction.

6. In paragraph 5, The above control unit, A driving robot characterized in that when checking for the presence or absence of an obstacle located in the first driving direction, if an obstacle exists in the first driving direction, it is checked whether the obstacle is a corridor wall, and if the obstacle is a corridor wall, the direction along the corridor wall is recognized as the direction of the destination.

7. In paragraph 6, The above control unit, A driving robot characterized in that when checking whether the obstacle is a corridor wall, if a straight line is detected in the first driving direction based on the surrounding environment information, the detected straight line is recognized as an obstacle, the detected straight line is checked to see if it is longer than a set length, and if the detected straight line is longer than the set length, the obstacle is recognized as a corridor wall.

8. In paragraph 1, The above neural network model is, A driving robot characterized in that it is pre-trained using a reinforcement learning method that determines a driving direction based on the direction information of the destination, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

9. In paragraph 8, The above neural network model is, A driving robot characterized in that it is pre-trained using a reinforcement learning method, which additionally receives noise information from a noise generator, determines a driving direction based on the noise information, direction information of the destination, distance information from the destination, and surrounding environment information, and provides a reward value or penalty value according to the determined driving direction.

10. In paragraph 8, The neural network model is, A driving robot characterized in that it is pre-trained using a reinforcement learning method in which a driving direction indicator is additionally input, a driving direction is determined based on the driving direction indicator, direction information of the destination, distance information from the destination, and surrounding environment information, and a reward value or penalty value is provided according to the determined driving direction.

11. In paragraph 8, The neural network model is, A driving robot characterized in that when the above-mentioned surrounding environment information is input, the surrounding environment information is preprocessed to be inverted left and right, the left and right inverted surrounding environment information is periodically learned, and the driving direction is determined based on the left and right inverted surrounding environment information, the direction information of the destination, the distance information to the destination, and the surrounding environment information, and a reward value or penalty value is provided according to the determined driving direction.

12. In paragraph 1, The above control unit, Includes a noise generator that generates noise information, A driving robot characterized in that, when noise information is obtained from the noise generator, the noise information, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information are input into the neural network model to determine a driving direction, thereby subjecting the neural network model to reinforcement learning.

13. In paragraph 1, The above control unit, A noise generator that generates noise information; and In order to pre-train the neural network model for rotation, a driving direction indicator generator is included that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator, A driving robot characterized in that, when noise information is obtained from the noise generator and a driving direction indicator is obtained from the driving direction indicator generator, the noise information, the driving direction indicator, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information are input into the neural network model to determine a driving direction, thereby subjecting the neural network model to reinforcement learning.

14. In paragraph 1, The above control unit, A noise generator that generates noise information; A driving direction indicator generator that generates driving direction indicators including a straight direction indicator, a right turn direction indicator, and a left turn direction indicator for pre-learning the neural network model for rotation; A replay buffer unit that collects and stores sensor information surrounding the robot used for learning the above neural network model; A preprocessing unit that extracts surrounding sensor information from the above replay buffer unit and preprocesses the surrounding sensor information so that it is inverted left and right; and It includes a learning unit that inputs the peripheral sensor information that has been inverted left and right into the neural network model and performs pre-training, A driving robot characterized in that when noise information is acquired from the noise generator, a driving direction indicator is acquired from the driving direction indicator generator, and a learning result value is acquired by the learning unit, the noise information, the driving direction indicator, learning result value information corresponding to the left-right inverted surrounding sensor information, result value information including a reward or penalty according to reinforcement learning, direction information of the destination, distance information from the destination, and surrounding environment information are input into the neural network model to perform reinforcement learning on the neural network model to determine a driving direction.

15. A method for controlling a driving robot including a driving motor unit, Step of acquiring surrounding environment information and map data; A step of confirming the robot location based on the above surrounding environment information and map data; A step of determining a driving method based on whether the above robot position is confirmed; If the driving method determined above is an artificial intelligence-based driving method due to a failure to confirm the robot's location, a step of inputting the direction information of the destination, the distance information to the destination, and the surrounding environment information into a pre-learned neural network model to determine the optimal driving direction; and A control method for a driving robot, characterized in that it includes a step of controlling the driving motor unit so that the driving robot moves in the determined driving direction.

Images