A delivery system and collision prevention method for a delivery robot

Abstract

The invention relates to a delivery system and a collision prevention method for a delivery robot (18), wherein an order (12) issued from an electronic platform (11 ) is transmitted via a cloud environment (16) as a delivery command (14); the delivery robot (18) being semi- autonomous or autonomous and assigned for delivery; a controller (20) located on the delivery robot (18) facilitating the movement of the robot with data flow to the cloud environment (16); the real-time status data set (22) of the delivery robot (18) being transmitted and stored via the cloud environment (16), and the orders (12) hosted by a central server (24) which is an infrastructure provider. The system includes a behavior planning unit (32) located on the delivery robot (18), which plans the robot's movements to ensure safe driving by detecting obstacles (30) in dense environments where dynamic obstacles (28) exist based on the images (26) received from the robot.

Description

[0001] A DELIVERY SYSTEM AND COLLISION PREVENTION METHOD FOR A DELIVERY ROBOT

[0002] TECHNICAL FIELD

[0003] The invention relates to a delivery system and a collision prevention method for a delivery robot in which movement planning is conducted.

[0004] STATE OF THE ART

[0005] Along with advancing technology, there is a great interest in making mobile delivery robots more intelligent. Delivery robots that operate semi-autonomously or autonomously need to have configurations that ensure vehicle safety and the safe transport of objects while preventing time and damage losses while they are in motion. Therefore, delivery robots are capable of planning their movements by detecting dynamic obstacles that may come in their way during delivery.

[0006] EP3371671 is related to a method, device, and assembly for map production. The invention describes a method of generating map data using a mobile robot system and straight lines extracted from visual images. Additionally, it describes a delivery robot with social navigation features that map for movement by analyzing image data from cameras.

[0007] BRIEF DESCRIPTION OF THE INVENTION

[0008] The object of the invention is to ensure the behavior planning of the robot by transferring some significant behaviors performed by moving obstacles for the safe driving of delivery robots in dense environments where dynamic obstacles exist.

[0009] To achieve the mentioned objective, the invention describes a delivery system for a delivery robot which includes a semi-autonomous or autonomous delivery robot responsible for making the delivery, which is transmitted over a cloud environment in response to an order from an electronic platform; a controller that allows robot movement and provides data flow with the cloud environment located on the delivery robot; a central server that includes an infrastructure provider for the orders where the current state data set of the delivery robot is transmitted and stored over the cloud environment. The invention includes a behavior planning unit that performs movement planning by detecting obstacles in dense environments with dynamic obstacles through images taken from the robot located on the delivery robot, thus providing safe driving by preventing collisions. In this way, it is provided that the robot makes human, object recognition and orientation estimation with the behavior planning unit, feeds the local planner, and performs social navigation. In this case, the safe driving of delivery robots in dense environments where dynamic obstacles exist can be ensured with the behavior planning unit. In other words, it prevents the delivery robots from hitting dynamic obstacles during the delivery process where they move according to the behavior planning unit.

[0010] In a preferred configuration of the invention, the behavior planning unit is set to provide mutual data flow with the controller. In this way, delivery robots can move according to social navigation provided by behavior planning.

[0011] In a preferred configuration of the invention, the behavior planning unit is set to perceive in accordance with the class and behavior modeling of dynamic obstacles. In this way, the features of dynamic obstacles are determined according to the models by image processing in images taken from the delivery robot. Also, the determination of the characteristics of dynamic obstacles, for example, the speed or direction of a dynamic (moving) obstacle, can make the robot's movement behavior planning.

[0012] A preferred configuration of the invention includes one or more depth cameras that capture images of dynamic obstacles providing forward vision angles to the robot located on the delivery robot. In this way, behavior planning is provided in the robot's forward movement direction by processing the images perceived from the depth cameras.

[0013] A preferred configuration of the invention includes one or more environmental cameras located in a position close to the corners of the environmental walls of the delivery robot and capturing images providing view angles from the robot's blind spots. In this way, behavior planning is provided in any direction of the robot's movement by processing the images perceived from the cameras on the environmental walls of the delivery robot. Here, the delivery robot is in a state that provides safe driving characteristics by imaging blind spots in all directions of the robot's movement.

[0014] In a preferred configuration of the invention, with the detection of dynamic obstacles through images in the behavior planning unit, the delivery robot is set to move away within 0.21 to 2 seconds, preferably 0.5 seconds. In this way, the delivery robot moves away from the obstacle in a time that it will not hit a dynamic obstacle that comes across it, for example.

[0015] In a preferred configuration of the invention, with the detection of dynamic obstacles through images in the behavior planning unit, the delivery robot is set to move a distance of 0.2 to 2 meters, preferably 1 meter. In this way, the delivery robot moves a distance that it will not hit a dynamic obstacle that comes across it, for example, by moving away from the obstacle.

[0016] A preferred application of the invention includes the steps of taking instant images from environmental cameras imaging blind spots on the delivery robot and depth cameras that can measure depth; classifying the positions of dynamic obstacles on the images taken from the cameras in the behavior planning unit with deep learning architectures; matching depth information with positions on classified images in the behavior planning unit; modeling the speed, orientation, and behavior of dynamic obstacles in the behavior planning unit; planning the robot's movements to prevent collisions according to the class and behavior models of approximately dynamic obstacles with the behavior planning unit. In this way, the robot's motion planning is ensured with the collision prevention method developed according to the delivery system, and the order is safely delivered.

[0017] In a preferred application of the invention, the behavior planning unit calculates the environmental risk factor according to the class and behavior models of dynamic obstacles and plans the linear speed of the robot. In this way, a risk factor for the collision situation is determined according to the visual data from the delivery robot, and the robot is brought to a speed that will move without hitting the obstacle.

[0018] BRIEF DESCRIPTION OF THE FIGURES

[0019] Figure 1 is a schematic display of a delivery system for a delivery robot.

[0020] Figure 2 shows a flowchart related to the collision prevention method for a delivery robot.

[0021] DETAILED DESCRIPTION OF THE INVENTION

[0022] In this detailed explanation, the subject of the invention development is described with references to examples, without any restrictions, merely to better explain the topic. Figure 1 schematically illustrates a delivery system for a delivery robot. In a delivery system (10) for a delivery robot, orders (12) are given from an electronic platform (11 ). Here, the electronic platform (11 ) can be an order site accessed from a computer or mobile device, or it can also be an order platform accessible via a mobile application. For instance, in response to an order (12) a customer gives, a delivery command (14) is formed on the electronic platform (11 ). The delivery command (14) is transmitted to a delivery robot (18) over a cloud environment (16). Here, the delivery robot (18) is a semi-autonomous or fully autonomous mobile robot tasked with delivering the received orders (12). Also, on the delivery robot (18), there is, for example, one or more wheels allowing forward and backward movement along the horizontal axis. In the subject matter of the invention, there are four wheels on a delivery robot (18). The delivery robot (18) has a suspension corresponding to each wheel that reduces vibration against obstacles, for example, bumps, pavements, etc., it encounters during delivery, allowing it to overcome the obstacles and maintain traction. Suspensions are mounted independently of each other. A controller (20), which operates the vehicle, i.e., provides the robot's motion by enabling the command control of the robot, is located on the delivery robot (18). Also, the controller (20) is an electronic circuit structure that allows data flow between the delivery robot (18) and the cloud environment (16). Here, the instantaneous state data (22) transmitted from the controller (20) to the cloud environment (16) during the order (12) delivery by the delivery robot (18) is transferred to a central server (24) over the cloud environment (16). These data (22) are stored in the central server (24). In the delivery system (10), the delivery robot (18) also includes a connection unit that provides internet access to the robot's controller (20) and the cloud environment (16). Also, the central server (24) is the infrastructure provider for the orders. In the delivery system (10) of the invention, there is a behavior planning unit (32) located on the delivery robot (18). The behavior planning unit (32) enables the detection (30) of obstacles in dense environments with dynamic obstacles (28) through images (26) taken from the robot. In the behavior planning unit (32), the detection (30) of obstacles prevents the delivery robots (18) from colliding and ensures safe driving by performing the robot's movement planning. Thus, the robot (18) is made to perform human, object recognition and orientation estimation (30), feeding the local planner. Here, social navigation is provided to the delivery robot (18) with behavior planning. In the delivery system (10), the behavior planning unit (32) provides mutual data flow with the controller (20). Thus, the delivery robots (18) can move according to the social navigation provided. In addition, the behavior planning unit (32) ensures the perception of dynamic obstacles (28) in accordance with the predetermined class modeling (34) and behavior modeling (36) through images (26). In this way, the features of dynamic obstacles (28) such as speed or position are determined. On each of the delivery robots (18), there is one or more depth cameras (38). Also, on the near positions (18) of the corners of the environmental walls of each delivery robot, there are one or more environmental cameras (40). Depth cameras (38) are electronic devices that receive the images (26) of dynamic obstacles (28) by providing forward viewing angles to the robot. Thus, with the image processing done, behavior planning is done in the forward movement direction of the robot (18). Environmental cameras (40) are electronic devices that receive images (26) by providing viewing angles from the blind spots of the robot (18). Therefore, with the image processing, behavior planning is done in any direction of the robot's movement (18). With the detection (30) of dynamic obstacles from images (26) in the behavior planning unit (32), the delivery robot (18) moves away from dynamic obstacles (28) within 0.21 to 2 seconds, for example, within 0.25 seconds or 0.5 seconds. Thus, the delivery robot (18) moves away from the obstacle at a time when it will not collide with the obstacle (28). Also, with the detection (30) of dynamic obstacles from images (26) in the behavior planning unit (32), the delivery robot (18) moves away from dynamic obstacles (28) a distance of 0.2 to 2 meters, for example, a distance of 1 meter or 1.5 meters or 1.8 meters. Thus, the delivery robot (18) moves to a distance where it will not collide with the obstacle (28) and moves away from the obstacle. With the social navigation in the invention, it is possible to navigate correctly in traffic with crowded people and obstacles by transferring the predicted decisions, such as a driver reducing speed when a pedestrian approaches the road while driving, to the behaviors of the robot (18), but there is no need to do this for a pedestrian walking along the pavement.

[0023] Figure 2 shows a flowchart related to the collision prevention method for a delivery robot. The collision prevention method for the delivery robot (18) is developed in accordance with the social navigation feature, aiming to provide the delivery system (10). The steps of this method, in their operating sequence, are as follows:

[0024] • First, the delivery robot (18) collects instantaneous images (42) from environmental cameras (40) that capture the robot's blind spots and depth cameras (38) capable of measuring depth.

[0025] • Subsequently, the positions of dynamic obstacles (28) within the images (26) collected by the cameras (38)(40) are determined and classified (48) in the behavior planning unit (32) using deep learning architectures (46).

[0026] • Next, in the behavior planning unit (32), the locations of classified (48) images (42) are matched (52) with depth information (50).

[0027] • Then, in the behavior planning unit (32), the speed, orientation, and behavior of the dynamic obstacles (28) are modeled (54).

[0028] • Finally, the robot's movements to prevent collisions are planned (56) in the behavior planning unit (32) according to the class and behavior models (34)(36) of the dynamic obstacles (28). In addition, in the delivery system and the delivery method for the subject matter delivery robot, an environmental risk factor is calculated (58) in the behavior planning unit (32) according to the class and behavior models (34)(36) of the dynamic obstacles (28). Depending on the calculation of the environmental risk factor (58), linear speed planning (60) of the robot (18) is performed. This way, a risk factor for the collision scenario of the robot (18) is determined and the robot reaches a speed at which it can move without colliding with the obstacle.

[0029] REFERENCE NUMBERS

[0030] 10 Delivery System 36 Behavior modeling

[0031] 11 Electronic platform 38 Depth camera

[0032] 12 Order 39 Environmental camera

[0033] 14 Delivery command 42 Instantaneous image capture

[0034] 16 Cloud environment 44 Positions of dynamic obstacles

[0035] 18 Delivery Robot 46 Deep learning architectures

[0036] 20 Controller 48 Classification of obstacle positions

[0037] 22 Instant state data 50 Depth information

[0038] 24 Central server 52 Matching of information

[0039] 26 Images 54 Modeling of obstacles

[0040] 28 Dynamic obstacles 56 Planning of robot movement

[0041] 30 Detection of obstacles 58 Calculation of risk factor

[0042] 32 Behavior planning unit 60 Linear speed planning

[0043] 34 Class modeling

Claims

CLAIMS1- A delivery system for a delivery robot (18), comprising semi-autonomous or autonomous a delivery robot (18) being assigned for delivery via an order (12) issued from an electronic platform (11 ) is transmitted via a cloud environment (16) as a delivery command (14); a controller (20) located on the delivery robot (18) facilitating the movement of the robot with data flow to the cloud environment (16); a central server (24) which is a infrastructure provider and by which the real-time status data set (22) of the delivery robot (18) being transmitted and stored via the cloud environment (16) with orders (12) hosted characterized in that a behavior planning unit (32) is disposed on the delivery robot (18) such that planning the robot's movement to avoid collisions by detecting obstacles (30) in dense environments where dynamic obstacles (28) exist based on the images (26) received from the robot.2- A delivery system for a delivery robot in accordance with Claim 1 , wherein the behavior planning unit (32) is configured to provide reciprocal data flow with the controller (20).3- A delivery system for delivery robots in accordance with any of the preceding claims, wherein the behavior planning unit (32) is configured to perceive dynamic obstacles (28) in accordance with the class and behavior models (34) (36) of the dynamic obstacles (28).4- A delivery system for delivery robots in accordance with any of the preceding claims, wherein one or more depth cameras (38) located on the delivery robot (18), capturing images (26) of dynamic obstacles (28) and providing front vision angles to the robot.5- A delivery system for delivery robots in accordance with any of the preceding claims, wherein one or more environmental cameras (40) located in close proximity to the corners of the environmental walls of the delivery robot (18), capturing images (26) and providing vision angles from the robot's (18) blind spots.6- A delivery system for delivery robots in accordance with any of the preceding claims, wherein being set to move away from dynamic obstacles (28) within 0.21 to 2 seconds, preferably within 0.5 seconds, upon detection (30) of dynamic obstacles (28) in the behavior planning unit (32) through images (26).7- A delivery system for delivery robots in accordance with any of the preceding claims, wherein being set to move away from dynamic obstacles (28) within a distance of 0.2 to 2meters, preferably 1 meter, upon detection (30) of dynamic obstacles (28) in the behavior planning unit (32) through images (26).8- A collision prevention method for delivery robots in accordance with any of the preceding claims, wherein receiving real-time images (42) from environmental cameras (40) that display blind spots on the delivery robot (18) and depth-measuring cameras (38); classifying (48) the positions (44) of dynamic obstacles (28) in the images (26) taken from the cameras (38) (40) in the behavior planning unit (32) using deep learning architectures (46); matching (52) the positions (44) in the acquired images (42) with depth information (50) in the behavior planning unit (32); modeling (54) the speed, orientation, and behavior of dynamic obstacles (28) in the behavior planning unit (32); and planning (56) the robot's movements to avoid collisions based on the class and behavior models (34) (36) of the dynamic obstacles (28) in the behavior planning unit (32). 9- A collision prevention method for delivery robots in accordance with any of the preceding claims, wherein the calculation of the environmental risk factor according to the class and behavior models (34) (36) of dynamic obstacles (28) in the behavior planning unit (18) and planning (60) the linear speed of the robot.

Images