Driving robot and control method thereof

Abstract

The present disclosure relates to a driving robot capable of moving in an optimal driving mode by selectively replacing a plurality of pre-trained artificial intelligence models according to a surrounding obstacle environment and a control method thereof. The driving robot may comprise: a sensor unit for acquiring surrounding environment information; an input unit for input of map data; a drive motor unit for moving the robot; and a control unit for controlling the drive motor unit by changing a driving mode in response to the surrounding environment information, wherein the control unit recognizes a surrounding obstacle situation on the basis of the map data and the surrounding environment information, selects one neural network model from among a plurality of pre-trained neural network models for each driving mode condition on the basis of the surrounding obstacle environment, switches a current driving mode to a driving mode based on the selected neural network model, and controls the drive motor unit to move the robot in the switched driving mode based on the neural network model.

Description

Driving robot and its control method

[0001] The present disclosure relates to a driving robot and a control method thereof that can move in an optimal driving mode by selectively replacing a plurality of pre-learned artificial intelligence models according to the surrounding obstacle environment.

[0002] In general, a robot is a machine that automatically processes or operates a given task with its own abilities, and the application fields of robots are generally classified into various fields such as industrial, medical, space, and underwater.

[0003] Recently, there has been an increasing trend of robots that can communicate or interact with humans through voice or gestures.

[0004] These robots can include various types of robots, such as guide robots that are placed in specific locations and provide various information to users, or home robots that are installed in homes.

[0005] In order for a robot to drive, it must find its location on a map, have a path to the destination location on the map, and be able to control its motors to follow that path.

[0006] The location of the robot on the map can be determined through the SLAM (Simultaneous Localization and Mapping) function, which compares the robot's sensor information, including camera images, lidar, and distances to obstacles around the robot, with map information.

[0007] In addition, the process of generating a path to the destination is divided into a process of generating a global path that generates a path from the robot location to the destination location on the map and a local path to avoid obstacles around the robot, and the robot can control the motor through a motion controller to follow the finally generated local path.

[0008] However, when the robot drives, it may drive well depending on the surrounding environment, but when passing through a narrow path or when there are many obstacles around, it may stop frequently or fail to drive.

[0009] In order for the robot to successfully drive in all environmental situations, it is necessary to adjust the parameters used to calculate the robot control values ​​as needed, but there was a problem that it was difficult to set the parameters according to all environmental situations.

[0010] Therefore, in the future, there is a need to develop a driving robot that can successfully drive to a destination without stopping or failing to drive even in an environment with surrounding obstacles while driving.

[0011] The present disclosure aims to solve the above-mentioned problems and other problems.

[0012] The present disclosure provides a driving robot and a control method thereof that can successfully drive to a destination without stopping or failing to drive even in an obstacle environment by selecting one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment and switching the current driving mode to a driving mode based on the selected neural network model.

[0013] According to one embodiment of the present disclosure, a driving robot includes a sensor unit for acquiring information about a surrounding environment, an input unit for inputting map data, a driving motor unit for moving the robot, and a control unit for controlling the driving motor unit by varying a driving mode in accordance with the information about the surrounding environment, wherein the control unit recognizes a surrounding obstacle situation based on the map data and the information about the surrounding environment, selects one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, switches the current driving mode to a driving mode based on the selected neural network model, and controls the driving motor unit so that the robot moves in the driving mode based on the switched neural network model.

[0014] A control method of a driving robot according to one embodiment of the present disclosure is a control method of a driving robot including a driving motor unit, and may include a step of acquiring map data and surrounding environment information, a step of recognizing a surrounding obstacle situation based on the map data and the surrounding environment information, a step of selecting one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, a step of switching a current driving mode to a driving mode based on the selected neural network model, and a step of controlling the driving motor unit so that the robot moves in the driving mode based on the switched neural network model.

[0015] According to one embodiment of the present disclosure, a driving robot can successfully drive to a destination without stopping or failing to drive even in an obstacle environment by selecting one of a plurality of pre-learned neural network models for each driving mode condition based on a surrounding obstacle environment and switching the current driving mode to a driving mode based on the selected neural network model.

[0016] FIG. 1 is a drawing for explaining a driving robot according to one embodiment of the present disclosure.

[0017] FIG. 2 is a drawing for explaining a process of forming an elliptical sensing area of ​​a driving robot according to an embodiment of the present disclosure.

[0018] FIG. 3 is a drawing for explaining a driving mode switching process of a driving robot according to one embodiment of the present disclosure.

[0019] FIG. 4 is a drawing for explaining a driving mode control process of a driving robot according to an embodiment of the present disclosure.

[0020] FIG. 5 and FIG. 6 are diagrams for explaining the learning process of a high-speed driving mode neural network model of a driving robot according to one embodiment of the present disclosure.

[0021] FIGS. 7 to 9 are diagrams for explaining the learning process of a neural network model for an obstacle avoidance driving mode of a driving robot according to an embodiment of the present disclosure.

[0022] FIG. 10 and FIG. 11 are diagrams for explaining the learning process of a neural network model for deadlock and collision overcoming mode of a driving robot according to one embodiment of the present disclosure.

[0023] Figure 12 is a flowchart for explaining a control process of a driving robot according to one embodiment of the present disclosure.

[0024] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present disclosure.

[0025] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0026] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0027] Additionally, throughout this specification, the terms "neural network," "neural network," and "network function" may be used interchangeably. A neural network may be comprised of a set of interconnected computational units, which may generally be referred to as "nodes." These "nodes" may also be referred to as "neurons." A neural network comprises at least two or more nodes. The nodes (or neurons) constituting the neural networks may be interconnected by one or more "links."

[0028] Artificial Intelligence (AI) refers to the field of study that studies artificial intelligence or the methodologies for creating it, while Machine Learning (ML) refers to the field that defines various problems in the field of AI and studies the methodologies for solving them. Machine learning is also defined as an algorithm that improves performance on a task through consistent experience.

[0029] An artificial neural network (ANN) is a model used in machine learning. It can refer to a model with problem-solving capabilities, comprised of artificial neurons (nodes) formed by the connection of synapses. An ANN can be defined by the connection patterns between neurons in different layers, the learning process that updates model parameters, and the activation function that generates output values.

[0030] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer may contain one or more neurons, and the artificial neural network may include synapses connecting neurons. In an artificial neural network, each neuron can output a function value of an activation function based on input signals, weights, and biases received through the synapses.

[0031] Model parameters are parameters determined through learning, including synaptic connection weights and neuron biases. Hyperparameters are parameters that must be set before learning in machine learning algorithms, including the learning rate, number of iterations, mini-batch size, and initialization function.

[0032] The goal of artificial neural network training can be seen as determining model parameters that minimize a loss function. The loss function can be used as an indicator for determining optimal model parameters during the artificial neural network training process.

[0033] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

[0034] Supervised learning refers to a method for training an artificial neural network given labels for training data. Labels can refer to the correct answer (or output) that the artificial neural network must infer when inputting training data. Unsupervised learning refers to a method for training an artificial neural network without given labels for the training data. Reinforcement learning refers to a learning method in which an agent defined within a given environment is trained to select actions or action sequences that maximize cumulative rewards in each state.

[0035] Among artificial neural networks, machine learning implemented with a deep neural network (DNN) containing multiple hidden layers is sometimes called deep learning, and deep learning is a subset of machine learning. Hereinafter, "machine learning" is used to encompass deep learning.

[0036] FIG. 1 is a drawing for explaining a driving robot according to one embodiment of the present disclosure.

[0037] As illustrated in FIG. 1, the driving robot (10) of the present disclosure may include a sensor unit (100) that acquires surrounding environment information, an input unit (200) that inputs map data, a drive motor unit (300) that moves the robot, and a control unit (400) that controls the drive motor unit (300) by varying the driving mode in accordance with the surrounding environment information.

[0038] Here, the sensor unit (100) may include at least one of a 3D camera, a lidar sensor, an ultrasonic sensor, a distance sensor, and a virtual sensor.

[0039] Additionally, the sensor unit (100) can provide obstacle information for an elliptical sensing area including a front obstacle proximity area, a front collision prediction area, and a rear obstacle proximity area.

[0040] The sensor unit (100) can increase the size of the forward obstacle proximity area among the sensing areas in proportion to the speed of the robot.

[0041] For example, the sensor unit (100) can increase the size of the front obstacle proximity area using a formula consisting of size of the front obstacle proximity area = robot surface distance + k1 × robot speed × L × │cos (θ)│^k2 + k3 (where L is a constant for determining the size of the front obstacle proximity area, θ is an angular difference from the front center, and k1, k2, and k3 are constants).

[0042] Additionally, the sensor unit (100) can provide a sensing area so that the size of the forward collision prediction area is smaller than the size of the forward obstacle proximity area.

[0043] In some cases, the sensor unit (100) may provide a sensing area so that the size of the rear obstacle proximity area is smaller than the size of the front collision prediction area and the size of the front obstacle proximity area.

[0044] In addition, the control unit (400) can recognize a surrounding obstacle situation based on map data and surrounding environment information, select one neural network model among a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, switch the current driving mode to a driving mode based on the selected neural network model, and control the driving motor unit (300) so that the robot moves in the driving mode based on the switched neural network model.

[0045] Here, when recognizing a surrounding obstacle situation, the control unit (400) can recognize a surrounding obstacle situation including the location of an obstacle located on the movement path to the destination based on map data and surrounding environment information.

[0046] Next, when selecting a neural network model, the control unit (400) may select a high-speed driving mode neural network model from among a plurality of pre-learned neural network models if no obstacle is detected within the forward obstacle proximity area based on the surrounding obstacle environment.

[0047] In addition, the control unit (400) may select an obstacle avoidance driving mode neural network model from among a plurality of pre-learned neural network models when an obstacle is detected within a front obstacle proximity area based on the surrounding obstacle environment and an obstacle is not detected within a front collision prediction area.

[0048] In addition, the control unit (400) can select a deadlock and collision overcoming mode neural network model among a plurality of pre-learned neural network models when an obstacle is detected within the forward collision prediction area based on the surrounding obstacle environment.

[0049] The high-speed driving mode neural network model can be pre-trained according to high-speed driving mode conditions when the surrounding obstacle environment is a high-speed driving mode condition.

[0050] For example, high-speed driving mode conditions may include a situation where the surrounding obstacle environment has a number of obstacles that is less than or equal to a standard number, the width of the driving path is greater than or equal to the standard width, and a turn toward the destination is possible at an intersection.

[0051] In addition, the high-speed driving mode neural network model can be trained using a reinforcement learning method that outputs robot speed and angular velocity information as result values ​​when sensor information, robot status information, target information, and mode information are input, and provides a reward value according to the result value.

[0052] Here, the sensor information may include surrounding environment information based on a lidar sensor, the robot status information may include speed and angular velocity information of the robot, the target information may include target direction and distance information of the robot, and the mode information may include straight-line mode information and rotation mode information.

[0053] For example, sensor information may include sensing of obstacle status up to a distance of 3m in front of the robot and surrounding environment information sensed and processed at 3-degree intervals.

[0054] And, the target information may include the following target angle information and target angle information to which random noise is added.

[0055] Additionally, the reward value may include a time reward value for reaching the goal in the shortest time, a learning assistance reward value for assisting learning, and a collision reduction reward value for reducing collisions with obstacles.

[0056] Here, the learning aid reward value may be a reward value provided when there is neither rotation nor velocity, when the robot is closer to the waypoint compared to the previous frame, or when the robot's direction is toward the target direction.

[0057] In addition, the high-speed driving mode neural network model can be trained to repeat the process of outputting the robot's speed and angular velocity information as result values ​​multiple times and to provide a reward value based on the final result value repeated multiple times.

[0058] Additionally, the high-speed driving mode neural network model can be trained based on input data including information on the environment of a turning learning path and information on the environment of a wide corridor with few obstacles.

[0059] Additionally, the high-speed driving mode neural network model can be trained based on left-right inverted data by inverting the simulation values ​​left-right so that left-right actions are learned evenly.

[0060] Next, the obstacle avoidance driving mode neural network model can be pre-trained according to obstacle avoidance driving mode conditions when the surrounding obstacle environment is an obstacle avoidance driving mode condition.

[0061] Here, the obstacle avoidance driving mode conditions may include a situation in which the surrounding obstacle environment has a number of obstacles greater than or equal to a standard number, the width of the driving path is less than or equal to the standard width, and driving is possible without collision on a narrow road.

[0062] The obstacle avoidance driving mode neural network model can be trained using a reinforcement learning method in which sensor information, robot status information, target information, and mode information are input, and the robot's speed, angular velocity, and center-of-path distance information are output as result values, and a reward value is provided according to the result value.

[0063] For example, sensor information may include surrounding environment information based on a lidar sensor, robot status information may include speed and angular velocity information of the robot, target information may include target direction and distance information of the robot, and mode information may include straight-line mode information and rotation mode information.

[0064] Here, the sensor information may include information on the surrounding environment sensed at 3-degree intervals and obstacle status sensed up to a distance of 1.5 m in front of the robot.

[0065] Additionally, the road center distance information may include distance information from the center of the robot to the center of the road.

[0066] Additionally, the reward value may include a path center reward value in which the robot is located in the center of the path, a path center correct reward value in which the output value of the path center distance information is close to the correct answer, a time reward value for reaching the goal in the shortest time, a learning assistance reward value for assisting learning, and a collision reduction reward value for reducing collisions with obstacles.

[0067] At this time, the learning assistance reward value may be a reward value provided when there is no rotation or velocity, when the robot is closer to the waypoint compared to the previous frame, or when the robot's direction is toward the target direction.

[0068] In addition, the obstacle avoidance driving mode neural network model can be trained to repeat the process of outputting the robot's speed and angular velocity information as result values ​​multiple times and to provide a reward value based on the final result value repeated multiple times.

[0069] Additionally, the obstacle avoidance driving mode neural network model can be trained based on input data including narrow road and maze environment information and narrow and obstacle-rich corridor environment information.

[0070] Additionally, the obstacle avoidance driving mode neural network model can be trained based on left-right inverted data by inverting the simulation values ​​left-right so that left-right actions are learned evenly.

[0071] Next, the deadlock and collision overcoming mode neural network model can be pre-trained according to the deadlock and collision overcoming mode conditions when the surrounding obstacle environment is a deadlock and collision overcoming mode condition.

[0072] Here, the deadlock and collision overcoming mode conditions may include a situation in which the surrounding obstacle environment is such that an obstacle is located in front of the robot, making forward movement impossible and requiring backward movement to secure another driving path.

[0073] The deadlock and collision overcoming mode neural network model can be trained using a reinforcement learning method that outputs robot speed and angular velocity information as result values ​​when sensor information, robot status information, target information, and rear sensor information are input, and a reward value is provided according to the result value.

[0074] Here, the sensor information may include surrounding environment information based on a lidar sensor, the robot status information may include speed and angular velocity information of the robot, the target information may include target direction and distance information of the robot, and the rear sensor information may include rear sensor data for avoiding collision when moving backward.

[0075] For example, sensor information may include sensing of obstacle status up to a distance of 1.5 m in front of the robot and surrounding environment information sensed and processed at 3-degree intervals.

[0076] Additionally, the reward value may include a deadlock check reward value that secures a path for the robot to drive, a drivable direction reward value that changes a direction in which the robot cannot pass into a direction in which the robot can pass by moving backwards and turning, and a time reward value for reaching the goal in the shortest time.

[0077] The deadlock and collision overcoming mode neural network model can perform reinforcement learning by deleting a reward value that hinders the robot from escaping from a deadlock among multiple reward values ​​during reinforcement learning.

[0078] Here, the interference compensation value may include a direction compensation value for the robot to move from a deadlock toward a goal and a distance compensation value for the robot to approach a waypoint.

[0079] In addition, the deadlock and collision overcoming mode neural network model can be trained to repeat the process of outputting the robot's speed and angular velocity information as result values ​​multiple times and to provide a reward value based on the final result value repeated multiple times.

[0080] Additionally, the deadlock and conflict resolution mode neural network model can be trained based on input data that includes deadlock environment information.

[0081] Additionally, the deadlock and collision overcoming mode neural network model can be trained based on left-right inverted data by inverting the simulation values ​​left-right so that left-right actions are learned evenly.

[0082] And, when switching to a driving mode based on a neural network model, the control unit (400) can switch the current driving mode from a high-speed driving mode neural network model-based driving mode to an obstacle avoidance driving mode neural network model-based driving mode, or switch from an obstacle avoidance driving mode neural network model-based driving mode to a high-speed driving mode neural network model-based driving mode.

[0083] Next, when switching to a driving mode based on a neural network model, the control unit (400) can switch the current driving mode from an obstacle avoidance driving mode based on a neural network model to a deadlock and collision overcoming mode based on a neural network model, or switch from a deadlock and collision overcoming driving mode based on a neural network model to an obstacle avoidance driving mode based on a neural network model.

[0084] In this way, the present disclosure selects one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, and switches the current driving mode to the driving mode based on the selected neural network model, thereby successfully driving to a destination without stopping or failing to drive even in an obstacle environment.

[0085] FIG. 2 is a drawing for explaining a process of forming an elliptical sensing area of ​​a driving robot according to an embodiment of the present disclosure.

[0086] As illustrated in FIG. 2, the driving robot (10) of the present disclosure can provide obstacle information for an elliptical sensing area including a front obstacle proximity area, a front collision prediction area, and a rear obstacle proximity area.

[0087] Here, the front obstacle proximity area of ​​the driving robot (10) can be defined as a curved area resembling an ellipse to better detect obstacles in front of the driving robot.

[0088] Additionally, the size of the curved shape resembling an ellipse is proportional to the robot speed (the greater the speed, the larger the ellipse's obstacle detection area), so that when the robot speed is faster, it can detect obstacles that are farther away.

[0089] In addition, the present disclosure may define a forward collision reservation area, which includes a space for recognizing whether an environment is difficult for the robot to drive in because an obstacle is too close to the robot, or a space where a collision with the robot is expected, as an area smaller than the forward obstacle proximity area.

[0090] In this way, the present disclosure can increase the size of the forward obstacle proximity area among the sensing areas in proportion to the speed of the robot.

[0091] For example, the present disclosure can increase the size of the front obstacle proximity area through a formula consisting of size of the front obstacle proximity area = robot surface distance + k1 × robot speed × L × │cos (θ)│^k2 + k3 (where L is a constant for determining the size of the front obstacle proximity area, θ is an angular difference from the front center, and k1, k2, and k3 are constants).

[0092] Additionally, the present disclosure can provide a sensing area so that the size of the forward collision prediction area is smaller than the size of the forward obstacle proximity area.

[0093] In some cases, the present disclosure may provide a sensing area such that the size of the rear obstacle proximity area is smaller than the size of the front collision prediction area and the size of the front obstacle proximity area.

[0094] FIG. 3 is a drawing for explaining a driving mode switching process of a driving robot according to one embodiment of the present disclosure.

[0095] As illustrated in FIG. 3, the present disclosure recognizes a surrounding obstacle situation based on map data and surrounding environment information, selects one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, switches the current driving mode to a driving mode based on the selected neural network model, and controls a drive motor so that the robot moves in the driving mode based on the switched neural network model.

[0096] The present disclosure can select a high-speed driving mode neural network model from among a plurality of pre-learned neural network models and switch to a driving mode based on the high-speed driving mode neural network model when an obstacle is not detected within a front obstacle proximity area based on the surrounding obstacle environment.

[0097] In addition, the present disclosure can select an obstacle avoidance driving mode neural network model from among a plurality of pre-learned neural network models and switch to a driving mode based on the obstacle avoidance driving mode neural network model when an obstacle is detected within a front obstacle proximity area based on the surrounding obstacle environment and an obstacle is not detected within a front collision prediction area.

[0098] In addition, the present disclosure can select a deadlock and collision overcoming mode neural network model from among a plurality of pre-learned neural network models when an obstacle is detected within a forward collision prediction area based on the surrounding obstacle environment, and switch to a driving mode based on the deadlock and collision overcoming mode neural network model.

[0099] In this way, the present disclosure can switch the current driving mode from a high-speed driving mode neural network model-based driving mode to an obstacle avoidance driving mode neural network model-based driving mode, or from an obstacle avoidance driving mode neural network model-based driving mode to a high-speed driving mode neural network model-based driving mode.

[0100] In addition, the present disclosure can switch the current driving mode from an obstacle avoidance driving mode neural network model-based driving mode to a deadlock and collision overcoming mode neural network model-based driving mode when switching to a driving mode based on a neural network model, or can switch from a deadlock and collision overcoming driving mode neural network model-based driving mode to an obstacle avoidance driving mode neural network model-based driving mode.

[0101] FIG. 4 is a drawing for explaining a driving mode control process of a driving robot according to an embodiment of the present disclosure.

[0102] As illustrated in FIG. 4, the present disclosure recognizes a surrounding obstacle situation based on map data and surrounding environment information, selects one neural network model among a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, switches the current driving mode to a driving mode based on the selected neural network model, and controls the robot to move in the driving mode based on the switched neural network model.

[0103] Here, the present disclosure can recognize a surrounding obstacle situation including the location of an obstacle located on a path to a destination based on map data and surrounding environment information when recognizing a surrounding obstacle situation.

[0104] For example, input data (350) may include sensor information, current status information, and target information.

[0105] Next, in the present disclosure, when selecting a neural network model, if no obstacle is detected within the forward obstacle proximity area based on the surrounding obstacle environment, a high-speed driving mode neural network model (310) is selected from among a plurality of pre-learned neural network models to control the robot to drive at high speed to a target point.

[0106] In addition, the present disclosure can control the robot to drive at low speed to a target point by selecting an obstacle avoidance driving mode neural network model (320) among a plurality of pre-learned neural network models when an obstacle is detected within a front obstacle proximity area based on the surrounding obstacle environment and an obstacle is not detected within a front collision prediction area.

[0107] In addition, the present disclosure can control the robot to drive at a low speed while avoiding backwards by selecting a deadlock and collision overcoming mode neural network model (330) from among a plurality of pre-learned neural network models when an obstacle is detected within a forward collision prediction area based on the surrounding obstacle environment.

[0108] FIG. 5 and FIG. 6 are diagrams for explaining the learning process of a high-speed driving mode neural network model of a driving robot according to one embodiment of the present disclosure.

[0109] As illustrated in FIG. 5, the high-speed driving mode neural network model (310) can be pre-learned according to high-speed driving mode conditions when the surrounding obstacle environment is a high-speed driving mode condition.

[0110] For example, high-speed driving mode conditions may include a situation where the surrounding obstacle environment has a number of obstacles that is less than or equal to a standard number, the width of the driving path is greater than or equal to the standard width, and a turn toward the destination is possible at an intersection.

[0111] In addition, the high-speed driving mode neural network model (310) can be trained using a reinforcement learning method in which, when sensor information, robot status information, target information, and mode information are input, the robot's speed and angular velocity information are output as result values, and a reward value is provided according to the result value.

[0112] Here, the sensor information may include surrounding environment information based on a lidar sensor, the robot status information may include speed and angular velocity information of the robot, the target information may include target direction and distance information of the robot, and the mode information may include straight-line mode information and rotation mode information.

[0113] For example, sensor information may include sensing of obstacle status up to a distance of 3m in front of the robot and surrounding environment information sensed and processed at 3-degree intervals.

[0114] And, the target information may include the following target angle information and target angle information to which random noise is added.

[0115] For example, the learning method of reinforcement learning is as follows.

[0116] Reinforcement learning is a method in which an agent (robot, in this case) takes actions (actions, in this case robot motion control) based on observations (observations, in this case information measured from sensors) in the surrounding environment, and receives rewards (rewards, if the action produces a good result, a reward, if it produces a bad result, a penalty; that is, if the robot avoids an obstacle and moves closer to the target point, a + score is given, and if the robot collides with an obstacle or moves away from an obstacle, a score is given) based on the results, and the agent (robot) learns to take actions (robot motion control) on its own to receive better rewards based on the environment (sensor information).

[0117] When training with reinforcement learning using an actual robot, training can be conducted primarily in a simulation environment, as incorrect movements of the robot can lead to accidents by colliding with obstacles.

[0118] In a simulation environment, a reinforcement learning motion controller can control a simulation robot to perform various actions.

[0119] As illustrated in FIG. 6, the present disclosure can be implemented as a simulation environment for reinforcement learning so as to reflect a rotational learning path environment and a wide, obstacle-free corridor environment as much as possible.

[0120] For example, as shown in Fig. 6, as a simulation environment for reinforcement learning, a rotational learning path environment can be simulated and used for learning, or a wide corridor environment with few obstacles can be simulated and used for learning.

[0121] And, the reward function may include a time reward value for reaching the goal in the shortest time, a learning assistance reward value for assisting learning, and a collision reduction reward value for reducing collisions with obstacles.

[0122] Here, the learning aid reward value may be a reward value provided when there is neither rotation nor velocity, when the robot is closer to the waypoint compared to the previous frame, or when the robot's direction is toward the target direction.

[0123] In addition, the high-speed driving mode neural network model (310) can be trained to repeat the process of outputting the robot's speed and angular velocity information as result values ​​multiple times and to provide a reward value based on the final result value repeated multiple times.

[0124] Additionally, the high-speed driving mode neural network model (310) can be trained based on input data including rotation learning path environment information and wide, obstacle-free corridor environment information.

[0125] In addition, the high-speed driving mode neural network model (310) can be trained based on left-right inversion data by inverting the simulation values ​​left-right so that left-right actions are learned evenly.

[0126] FIGS. 7 to 9 are diagrams for explaining the learning process of a neural network model for an obstacle avoidance driving mode of a driving robot according to an embodiment of the present disclosure.

[0127] As illustrated in FIG. 7, the obstacle avoidance driving mode neural network model (320) can be pre-learned according to obstacle avoidance driving mode conditions when the surrounding obstacle environment is an obstacle avoidance driving mode condition.

[0128] Here, the obstacle avoidance driving mode conditions may include a situation in which the surrounding obstacle environment has a number of obstacles greater than or equal to a standard number, the width of the driving path is less than or equal to the standard width, and driving is possible without collision on a narrow road.

[0129] The obstacle avoidance driving mode neural network model (320) can be trained using a reinforcement learning method in which, when sensor information, robot status information, target information, and mode information are input, the robot's speed, angular velocity, and road center distance information are output as result values, and a reward value is provided according to the result value.

[0130] For example, sensor information may include surrounding environment information based on a lidar sensor, robot status information may include speed and angular velocity information of the robot, target information may include target direction and distance information of the robot, and mode information may include straight-line mode information and rotation mode information.

[0131] Here, the sensor information may include information on the surrounding environment sensed at 3-degree intervals and obstacle status sensed up to a distance of 1.5 m in front of the robot.

[0132] Additionally, the road center distance information may include distance information from the center of the robot to the center of the road.

[0133] As shown in Fig. 8, the road center distance information can be output including the distance from the center of the driving robot (10) to the center of the road, in order to have the center information within the network.

[0134] As illustrated in FIG. 9, the present disclosure can be implemented as a simulation environment for reinforcement learning to reflect narrow path and maze environments and narrow, obstacle-filled corridor environments as much as possible.

[0135] For example, as shown in Fig. 9, as a simulation environment for reinforcement learning, a narrow path and maze environment can be simulated and used for learning, or a narrow corridor environment with many obstacles can be simulated and used for learning.

[0136] Additionally, the reward function may include a path center reward value in which the robot is located in the center of the path, a path center correct reward value in which the output value of the path center distance information is close to the correct answer, a time reward value in which the robot reaches the goal in the shortest time, a learning assistance reward value in which the robot assists learning, and a collision reduction reward value in which the robot reduces collisions with obstacles.

[0137] At this time, the learning assistance reward value may be a reward value provided when there is no rotation or velocity, when the robot is closer to the waypoint compared to the previous frame, or when the robot's direction is toward the target direction.

[0138] In addition, the obstacle avoidance driving mode neural network model (320) can be trained to repeat the process of outputting the robot's speed and angular velocity information as result values ​​multiple times and to provide a reward value based on the final result value repeated multiple times.

[0139] Additionally, the obstacle avoidance driving mode neural network model (320) can be trained based on input data including narrow road and maze environment information and narrow and obstacle-filled corridor environment information.

[0140] In addition, the obstacle avoidance driving mode neural network model (320) can be trained based on left-right inversion data so that left-right actions are learned evenly by inverting the simulation values ​​left-right.

[0141] FIG. 10 and FIG. 11 are diagrams for explaining the learning process of a neural network model for deadlock and collision overcoming mode of a driving robot according to one embodiment of the present disclosure.

[0142] As illustrated in FIG. 10, the deadlock and collision overcoming mode neural network model (330) can be pre-trained according to deadlock and collision overcoming mode conditions when the surrounding obstacle environment is a deadlock and collision overcoming mode condition.

[0143] Here, the deadlock and collision overcoming mode conditions may include a situation in which the surrounding obstacle environment is such that an obstacle is located in front of the robot, making forward movement impossible and requiring backward movement to secure another driving path.

[0144] The deadlock and collision overcoming mode neural network model (330) can be trained using a reinforcement learning method in which, when sensor information, robot status information, target information, and rear sensor information are input, the robot's speed and angular velocity information are output as result values, and a reward value is provided according to the result value.

[0145] Here, the sensor information may include surrounding environment information based on a lidar sensor, the robot status information may include speed and angular velocity information of the robot, the target information may include target direction and distance information of the robot, and the rear sensor information may include rear sensor data for avoiding collision when moving backward.

[0146] For example, sensor information may include sensing of obstacle status up to a distance of 1.5 m in front of the robot and surrounding environment information sensed and processed at 3-degree intervals.

[0147] As illustrated in Fig. 11, the present disclosure can be implemented as a simulation environment for reinforcement learning to reflect a deadlocked state environment as much as possible.

[0148] For example, as shown in Fig. 9, a deadlocked state environment can be simulated and used for learning as a simulation environment for reinforcement learning.

[0149] Additionally, the reward function may include a deadlock check reward value that secures a path for the robot to drive, a drivable direction reward value that changes a direction in which the robot cannot pass into a direction in which the robot can pass by moving backwards and turning, and a time reward value for reaching the goal in the shortest time.

[0150] The deadlock and collision overcoming mode neural network model (330) can perform reinforcement learning by deleting a reward value that hinders the robot from escaping from a deadlock among multiple reward values ​​during reinforcement learning.

[0151] Here, the interference compensation value may include a direction compensation value for the robot to move from a deadlock toward a goal and a distance compensation value for the robot to approach a waypoint.

[0152] In addition, the deadlock and collision overcoming mode neural network model (330) can be trained to repeat the process of outputting the robot's speed and angular velocity information as result values ​​multiple times and to provide a reward value based on the final result value repeated multiple times.

[0153] Additionally, the deadlock and collision overcoming mode neural network model (330) can be learned based on input data including deadlock state environment information.

[0154] In addition, the deadlock and collision overcoming mode neural network model (330) can be trained based on left-right inversion data so that left-right actions are evenly learned by inverting simulation values ​​left-right.

[0155] Figure 12 is a flowchart for explaining a control process of a driving robot according to one embodiment of the present disclosure.

[0156] As illustrated in FIG. 12, the present disclosure can acquire map data and surrounding environment information when the robot drives (S10), and recognize surrounding obstacle situations based on the map data and surrounding environment information (S20).

[0157] In addition, the present disclosure selects one neural network model among a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, switches the current driving mode to a driving mode based on the selected neural network model, and controls the drive motor so that the robot moves in the driving mode based on the switched neural network model.

[0158] In the present disclosure, if an obstacle is not detected within a front obstacle proximity area based on the surrounding obstacle environment, a high-speed driving mode neural network model can be selected from among a plurality of pre-learned neural network models to switch to a driving mode based on the high-speed driving mode neural network model (S30).

[0159] In addition, the present disclosure can select an obstacle avoidance driving mode neural network model from among a plurality of pre-learned neural network models and switch to a driving mode based on the obstacle avoidance driving mode neural network model when an obstacle is detected within a front obstacle proximity area based on the surrounding obstacle environment and no obstacle is detected within a front collision prediction area (S40).

[0160] In addition, the present disclosure can select a deadlock and collision overcoming mode neural network model among a plurality of pre-learned neural network models when an obstacle is detected within a forward collision prediction area based on the surrounding obstacle environment, and switch to a driving mode based on the deadlock and collision overcoming mode neural network model (S50).

[0161] In this way, the present disclosure can switch the current driving mode from a high-speed driving mode neural network model-based driving mode to an obstacle avoidance driving mode neural network model-based driving mode, or from an obstacle avoidance driving mode neural network model-based driving mode to a high-speed driving mode neural network model-based driving mode.

[0162] In addition, the present disclosure can switch the current driving mode from an obstacle avoidance driving mode neural network model-based driving mode to a deadlock and collision overcoming mode neural network model-based driving mode when switching to a driving mode based on a neural network model, or can switch from a deadlock and collision overcoming driving mode neural network model-based driving mode to an obstacle avoidance driving mode neural network model-based driving mode.

[0163] In this way, the present disclosure selects one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, and switches the current driving mode to the driving mode based on the selected neural network model, thereby successfully driving to a destination without stopping or failing to drive even in an obstacle environment.

[0164] According to the driving robot according to the present disclosure, by selecting one of a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment and switching the current driving mode to the driving mode based on the selected neural network model, the driving robot can successfully drive to the destination without stopping or failing to drive even in an obstacle environment, and thus has significant industrial applicability.

Claims

1. Sensor unit that obtains information about the surrounding environment; Input section where map data is entered; A drive motor that moves the robot; and It includes a control unit that controls the driving motor unit by changing the driving mode in accordance with the above-mentioned surrounding environment information, The above control unit, A driving robot characterized in that it recognizes a surrounding obstacle situation based on the above map data and the surrounding environment information, selects one neural network model among a plurality of pre-learned neural network models for each driving mode condition based on the surrounding obstacle environment, switches the current driving mode to a driving mode based on the selected neural network model, and controls the driving motor unit so that the robot moves in the driving mode based on the switched neural network model.

2. In paragraph 1, The above control unit, A driving robot characterized in that, when recognizing the above-mentioned surrounding obstacle situation, the driving robot recognizes the surrounding obstacle situation including the location of the obstacle located on the movement path to the destination based on the map data and the surrounding environment information.

3. In paragraph 1, The above control unit, When selecting the above neural network model, if no obstacle is detected within the forward obstacle proximity area based on the surrounding obstacle environment, a high-speed driving mode neural network model is selected from among the plurality of pre-learned neural network models. Based on the above surrounding obstacle environment, if an obstacle is detected within the forward obstacle proximity area and no obstacle is detected within the forward collision prediction area, an obstacle avoidance driving mode neural network model is selected from among the plurality of pre-learned neural network models. A driving robot characterized in that, when an obstacle is detected within a forward collision prediction area based on the above-mentioned surrounding obstacle environment, a deadlock and collision overcoming mode neural network model is selected from among the plurality of pre-learned neural network models.

4. In paragraph 3, The above high-speed driving mode neural network model is, A driving robot characterized in that, when the above-mentioned surrounding obstacle environment is a high-speed driving mode condition, pre-learning is performed according to the high-speed driving mode condition.

5. In paragraph 4, The above high-speed driving mode conditions are: A driving robot characterized in that the above surrounding obstacle environment includes a situation in which the number of obstacles is less than or equal to a standard number, the width of the driving path is greater than or equal to the standard width, and a turning toward the destination is possible at an intersection.

6. In paragraph 3, The above high-speed driving mode neural network model is, A driving robot characterized in that, when sensor information, robot status information, target information, and mode information are input, the robot's speed and angular velocity information are output as result values, and a reward value is provided according to the result values, through reinforcement learning.

7. In paragraph 3, The above obstacle avoidance driving mode neural network model is, A driving robot characterized in that, when the above-mentioned surrounding obstacle environment is an obstacle avoidance driving mode condition, the driving robot is pre-learned according to the obstacle avoidance driving mode condition.

8. In paragraph 7, The above obstacle avoidance driving mode conditions are: A driving robot characterized in that the above surrounding obstacle environment includes a situation in which the number of obstacles is greater than or equal to a standard number, the width of the driving path is less than or equal to the standard width, and driving is possible without collision in a narrow path.

9. In paragraph 3, The above obstacle avoidance driving mode neural network model is, A driving robot characterized in that, when sensor information, robot status information, target information, and mode information are input, the robot's speed, angular velocity, and path center distance information are output as result values, and a reward value is provided according to the result values, through reinforcement learning.

10. In paragraph 3, The above deadlock and conflict resolution mode neural network model is, A driving robot characterized in that, when the above-mentioned surrounding obstacle environment is a deadlock and collision overcoming mode condition, the driving robot is pre-learned according to the deadlock and collision overcoming mode condition.

11. In paragraph 10, The above deadlock and conflict resolution mode conditions are: A driving robot characterized in that the above-mentioned surrounding obstacle environment includes a situation in which an obstacle is located in front of the robot, making forward movement impossible and requiring backward movement to secure a different driving path.

12. In paragraph 3, The above deadlock and conflict resolution mode neural network model is, A driving robot characterized in that, when sensor information, robot status information, target information, and rear sensor information are input, the robot's speed and angular velocity information are output as result values, and a reward value is provided according to the result values, through reinforcement learning.

13. In paragraph 1, The above control unit, A driving robot characterized in that, when switching to the driving mode based on the neural network model, the current driving mode is switched from the high-speed driving mode neural network model-based driving mode to the obstacle avoidance driving mode neural network model-based driving mode, or from the obstacle avoidance driving mode neural network model-based driving mode to the high-speed driving mode neural network model-based driving mode.

14. In paragraph 1, The above control unit, A driving robot characterized in that, when switching to the driving mode based on the neural network model, the current driving mode is switched from the obstacle avoidance driving mode based on the neural network model to the deadlock and collision overcoming mode based on the neural network model, or from the deadlock and collision overcoming driving mode based on the neural network model to the obstacle avoidance driving mode based on the neural network model.

15. A method for controlling a driving robot including a driving motor unit, A step of acquiring the above map data and the above surrounding environment information; A step of recognizing a surrounding obstacle situation based on the above map data and the surrounding environment information; A step of selecting one neural network model among a plurality of pre-trained neural network models for each driving mode condition based on the above surrounding obstacle environment; A step of switching the current driving mode to a driving mode based on the selected neural network model; and A control method for a driving robot, characterized in that it comprises a step of controlling the driving motor unit so that the robot moves in a driving mode based on the converted neural network model.

Images