Obstacle detection device, control device, obstacle detection method, control method, and program

Abstract

To maintain the continuity of identification numbers of detected obstacles even when the obstacles are not detected temporarily. [Solution] The obstacle detection device includes a depth information acquisition unit that acquires depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot, an obstacle detection unit that detects one or more obstacles present in the travel path based on the depth information, and an obstacle management unit that manages the identification numbers of obstacles based on a first obstacle list containing information on detected obstacles and a second obstacle list containing information on obstacles that have already been detected. The second obstacle list includes, for each identification number, a non-update count that indicates the time elapsed since the obstacle was last detected. The obstacle management unit deletes from the second obstacle list any obstacle whose non-update count indicates a time longer than a predetermined time.

Description

[Technical Field]

[0001] The present disclosure relates to an obstacle detection device, a control device, an obstacle detection method, a control method, and a program. [Background technology]

[0002] As a related technique, Patent Document 1 discloses a control device for a self-propelled mobile body. In Patent Document 1, the self-propelled mobile body moves toward a target position while avoiding obstacles. An obstacle sensor such as a Positioning Sensitive Detector (PSD) camera is installed on the ceiling surface of the area in which the self-propelled mobile body travels. The obstacle sensor sequentially detects the position of obstacles moving on the plane in which the self-propelled mobile body moves. Based on the position information of the obstacles, the control device generates a stochastic potential field that represents the probability that the obstacle may exist. Based on the generated stochastic potential field, the control device searches for a route for the self-propelled mobile body to move toward the target position and moves the mobile body along the searched route.

[0003] As another related technique, Patent Document 2 discloses a collision avoidance system for work vehicles. In Patent Document 2, a vehicle such as a tractor has an obstacle detection unit. The obstacle detection unit includes a Light Detection and Ranging (LiDAR) sensor. The LiDAR sensor emits measurement light from the vehicle body toward a group of ranging points present within a predetermined measurement range and receives light reflected from the measurement light. The obstacle detection unit measures multiple distance values ​​for each ranging point based on the measurement light and the reflected light. The obstacle detection unit determines the presence or absence of an obstacle based on measurement information including the multiple measured distance values.

[0004] In Patent Document 2, the obstacle detection unit uses a grid map to identify the position of an obstacle. The grid map has a number of grids obtained by dividing the measurement range of the LiDAR sensor at a predetermined resolution. The obstacle detection unit identifies the highest height information in each grid from the coordinates of multiple measurement points in the up-down direction of the tractor, which are included in the point cloud information of each grid, and registers each identified height information as height information for that grid.

[0005] The obstacle detection unit extracts feature points for identifying obstacles. The obstacle detection unit also extracts candidate grids in which point cloud information that may indicate obstacles is registered. Of the candidate grids, the obstacle detection unit identifies candidate grids in which point cloud information having feature points for identifying obstacles exists as obstacle grids. The obstacle detection unit groups consecutive obstacle grids from the identified plurality of obstacle grids as a single obstacle. If the difference between the center position of the obstacle grid group for the past three steps and the center position of the current obstacle grid group is less than a predetermined threshold, the obstacle detection unit determines that the obstacle detected in the past and the obstacle detected in the current frame are the same obstacle. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-241836 [Patent Document 2] Patent No. 7402608 Summary of the Invention [Problem to be solved by the invention]

[0007] In Patent Documents 1 and 2, only information about current or very recent obstacles is stored. As a result, in Patent Documents 1 and 2, the continuity of detected obstacles over time may be interrupted due to fluctuations in depth values ​​caused by sensor measurement errors. For example, if two obstacles are detected at a certain timing, the two detected obstacles may be erroneously detected as a single obstacle at the next timing. Furthermore, if the two obstacles are correctly detected at the next timing, a new identification number is assigned to the detected obstacle, and the obstacle is detected as a new obstacle.

[0008] One of the objectives of the present disclosure is to provide an obstacle detection device, a control device, an obstacle detection method, a control method, and a program that can maintain the continuity of the identification numbers of detected obstacles even when an obstacle is temporarily not detected. [Means for solving the problem]

[0009] An obstacle detection device according to a first aspect of the present disclosure includes: a depth information acquisition unit that acquires depth information of a travel path of a mobile robot based on sensor data from a sensor that senses the travel path; an obstacle detection unit that detects one or more obstacles present in the travel path based on the depth information; and an obstacle management unit that manages identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected. The second obstacle list includes, for each identification number, a non-update count that indicates the time elapsed since the obstacle was last detected, and the obstacle management unit deletes from the second obstacle list any obstacle for which the non-update count indicates a time longer than a predetermined time.

[0010] A control device according to a second aspect of the present disclosure includes the obstacle detection device described above and a control unit that controls the mobile robot based on information about obstacles included in the second obstacle list.

[0011] An obstacle detection method according to a third aspect of the present disclosure includes: acquiring depth information of a travel path of a mobile robot based on sensor data from a sensor that senses the travel path; detecting one or more obstacles present in the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including a non-update count indicating the time elapsed since the obstacle was last detected for each identification number of the obstacle; and deleting from the second obstacle list any obstacle for which the non-update count indicates a time longer than a predetermined time.

[0012] A control method according to a fourth aspect of the present disclosure includes: acquiring depth information of a travel path of a mobile robot based on sensor data from a sensor that senses the travel path; detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information of the detected obstacles and a second obstacle list including information of obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; deleting from the second obstacle list any obstacle for which the non-update count indicates a time longer than a predetermined time; and controlling the mobile robot based on the obstacle information included in the second obstacle list.

[0013] A program according to a fifth aspect of the present disclosure causes a computer to perform the following process: acquire depth information of a travel path of a mobile robot based on sensor data from a sensor that senses the travel path; detect one or more obstacles present in the travel path based on the depth information; manage the identification numbers of the obstacles based on a first obstacle list containing information of the detected obstacles and a second obstacle list containing information of obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; and delete from the second obstacle list any obstacle whose non-update count indicates a time longer than a predetermined time.

[0014] A program according to a sixth aspect of the present disclosure causes a computer to perform processing including: acquiring depth information of a travel path of a mobile robot based on sensor data from a sensor that senses the travel path; detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on information included in a first obstacle list containing information of the detected obstacles and a second obstacle list containing information of obstacles that have already been detected, the second obstacle list including, for each of the obstacle identification numbers, a non-update count indicating the time elapsed since the obstacle was last detected; deleting from the second obstacle list any obstacle whose non-update count indicates a time longer than a predetermined time; and controlling the mobile robot based on the obstacle information included in the second obstacle list. [Effects of the Invention]

[0015] The obstacle detection device, control device, obstacle detection method, control method, and program disclosed herein can maintain the continuity of the identification numbers of detected obstacles even if an obstacle is temporarily not detected. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 2 is a block diagram illustrating a schematic configuration example of a control device according to the present disclosure. [Figure 2] 1 is a schematic diagram illustrating an example configuration of a system including a control device according to the present disclosure. [Figure 3] FIG. 2 is a block diagram illustrating an example of the configuration of a control server. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of an obstacle detection unit. [Figure 5] FIG. 10 is a diagram showing a specific example of a list of obstacles detected by an obstacle detection unit. [Figure 6] 10 shows a list of detected obstacles managed by an obstacle management unit. [Figure 7] 10 is a flowchart showing an operation procedure of the control server. [Figure 8] FIG. 1 is a block diagram illustrating an example of the configuration of a computer device. DETAILED DESCRIPTION OF THE INVENTION

[0017] Prior to describing embodiments of the present disclosure, an overview of the present disclosure will be described. Fig. 1 is a block diagram showing a schematic configuration example of a control device according to the present disclosure. The control device 10 shown in Fig. 1 includes a depth information acquisition unit 11, an obstacle detection unit 12, an obstacle management unit 13, and a control unit 14. In the control device 10, the depth information acquisition unit 11, the obstacle detection unit 12, and the obstacle management unit 13 correspond to an obstacle detection device 20.

[0018] The depth information acquisition unit 11 acquires depth information of the traveling path based on sensor data from a sensor that senses the traveling path of the mobile robot. The obstacle detection unit 12 detects one or more obstacles that exist on the traveling path of the mobile robot based on the acquired depth information. The obstacle management unit 13 manages the identification numbers of obstacles based on a first obstacle list that includes information on detected obstacles and a second obstacle list that includes information on obstacles that have already been detected.

[0019] The second obstacle list includes, for each identification number, a non-update count indicating the time elapsed since the obstacle was last detected. The obstacle management unit deletes from the second obstacle list any obstacle whose non-update count indicates a time longer than a predetermined time. The control unit 14 controls the mobile robot based on the information about the obstacle included in the second obstacle list.

[0020] In the present disclosure, the obstacle management unit 13 manages the identification numbers of obstacles based on the first obstacle list and the second obstacle list. The obstacle management unit 13 also deletes from the second obstacle list any obstacle whose non-update count is longer than a predetermined time. In this manner, the obstacle management unit 13 can prevent the number of obstacles included in the second obstacle list from becoming excessively large, and can assign the same identification number to the same obstacle even if the obstacle temporarily disappears. The control device 10 controls the mobile robot based on information about the obstacles detected by the obstacle detection device 20, thereby enabling the mobile robot to travel safely.

[0021] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the following description and drawings have been omitted and simplified as appropriate for clarity of explanation. In addition, in the following drawings, the same or similar elements are designated by the same reference numerals, and duplicate explanations are omitted as necessary.

[0022] FIG. 2 is a schematic diagram showing an example configuration of a system including a control device according to the present disclosure. In the following embodiment, the control device is configured as a control server 110. The system 100 shown in FIG. 2 includes the control server 110, a sensor 130, and a mobile robot 150. Note that FIG. 2 illustrates one sensor 130 and one mobile robot 150. However, the number of sensors 130 and the number of mobile robots 150 are not limited to one each. The system 100 may include multiple sensors 130. The system 100 may also include multiple mobile robots 150.

[0023] The mobile robot 150 has, for example, a power source such as a motor, a drive mechanism, and a wireless communication device. The mobile robot 150 travels within a predetermined movement range 200. The movement range 200 is also called the travel path of the mobile robot 150. The mobile robot 150 is configured as a transport robot that transports luggage in a warehouse, for example. The mobile robot 150 is configured to be capable of wireless communication with the control server 110. The mobile robot 150 may be configured to be capable of autonomous movement within the movement range 200.

[0024] The sensor 130 is a sensor that senses the movement range of the mobile robot 150. The sensor 130 is installed, for example, on the ceiling of the movement range 200 of the mobile robot 150. The movement range 200 may be divided into multiple areas, and a sensor 130 may be installed in each divided area. The sensor 130 includes an imaging device that can acquire depth information. The sensor 130 is used to acquire image information and depth information. In the following description, an example in which the sensor 130 includes a stereo camera will be mainly described. The sensor 130 transmits sensor data, for example, image data from the stereo camera, to the control server 110.

[0025] The control server 110 acquires image data from the sensor 130. Based on the acquired image data, the control server 110 detects obstacles present in the movement range 200. Based on information about the detected obstacles, the control server 110 directly or indirectly controls the movement of the mobile robot 150. For example, the control server 110 controls the movement of the mobile robot 150 by transmitting information about the detected obstacles to the mobile robot 150. Alternatively, the control server 110 may generate movement path information for the mobile robot 150 according to the detected obstacles and transmit the movement path information to the mobile robot 150 to control the movement of the mobile robot 150.

[0026] 3 is a block diagram showing an example configuration of the control server 110. The control server 110 has a depth information acquisition unit 111, an obstacle detection unit 112, an obstacle management unit 113, and an obstacle information transmission unit 114. In the control server 110, the depth information acquisition unit 111, the obstacle detection unit 112, and the obstacle management unit 113 configure an obstacle detection device 115. The control server 110 corresponds to the control device 10 shown in FIG. 1. The obstacle detection device 115 corresponds to the obstacle detection device 20 shown in FIG. 1.

[0027] The control server 110 may be physically configured using a computer device having one or more memories and one or more processors. In the control server 110, the one or more processors may execute processing in accordance with instructions read from the one or more memories, thereby realizing at least a portion of the functions of each unit in the control server 110. In this embodiment, the control server 110 does not necessarily have to be configured as a single computer device. The control server 110 may also be configured using multiple physically separated devices.

[0028] The depth information acquisition unit 111 acquires depth information of the movement range 200 of the mobile robot 150 based on image data acquired from the sensor 130. Here, the depth information indicates, for example, the height from the floor surface on which the mobile robot 150 moves. For example, in FIG. 2, when the mobile robot 150 moves on the xy plane, the depth information indicates the height in the z direction at each location in the movement range 200. The depth information acquisition unit 111 acquires, for example, point cloud values ​​(x, y, z) for pixels in the imaging area of ​​a stereo camera, and acquires depth information based on the acquired point cloud values. The depth information acquisition unit 111 may acquire image information including the brightness of each coordinate in addition to the depth information. The depth information acquisition unit 111 corresponds to the depth information acquisition unit 11 shown in FIG. 1.

[0029] The obstacle detection unit 112 detects one or more obstacles present in the movement range 200 based on the depth information acquired by the depth information acquisition unit 111. The obstacle detection unit 112 generates a grid map obtained by dividing the movement range 200 into a plurality of grids. The obstacle detection unit 112 identifies, in the grid map, grids whose depth information is at or above a predetermined height as obstacle grids. Furthermore, the obstacle detection unit 112 groups the identified obstacle grids into one or more obstacle grid groups, and detects the grouped obstacle grid groups as obstacles. The obstacle detection unit 112 corresponds to the obstacle detection unit 12 shown in FIG. 1.

[0030] Fig. 4 is a block diagram showing an example configuration of the obstacle detection unit 112. In the example of Fig. 4, the obstacle detection unit 112 includes a grid map generation unit 121, a grid attribute determination unit 122, and a grouping unit 123. The grid map generation unit 121 obtains a representative value of the point cloud z values ​​of the corresponding pixel values ​​for each grid, i.e., for each area, in each of a plurality of areas obtained by dividing the movement range 200 into a grid. For each grid, the grid map generation unit 121 sets the representative value of the point cloud z values ​​as the depth information of the grid.

[0031] The grid map generation unit 121 may calculate, for each grid, the average value of the point cloud z values ​​of the pixels contained in the grid, and use the calculated average value as the depth information of the grid. Alternatively, the grid map generation unit 121 may use the maximum value of the point cloud z values ​​as the depth information of the grid. The number of pixels contained in one grid depends on the number of divided areas, i.e., the number of grids. The grid map generation unit 121 may calculate, for each grid, the difference in maximum brightness between the previous image information and the current image information, and use the difference in maximum brightness as the brightness difference of the grid.

[0032] The grid map generating unit 121 may perform a smoothing process on the depth information. For example, the grid map generating unit 121 may set a smoothing coefficient k (0≦k<1) and use (depth information in the immediately preceding frame)×k+(depth information in the current frame)×(1−k) as the depth information in the current frame. When performing the smoothing process, the grid map generating unit 121 may control the smoothing coefficient k according to the luminance difference between the grids. Note that the “current frame” refers to the frame currently being processed, and does not necessarily refer to the frame at the time of processing execution or the latest frame.

[0033] For example, a grid having a brightness difference greater than a first threshold is considered to contain a dynamic object, i.e., a dynamic obstacle. In this case, the grid map generator 121 may set the smoothing coefficient to a relatively small value, for example, k=0.3. After setting the smoothing coefficient to k=0.3, the grid map generator 121 may maintain the smoothing coefficient at k=0.3 until a predetermined number of frames, for example, 90 frames, have elapsed.

[0034] On the other hand, a grid where the brightness difference is smaller than the second threshold is considered to contain a static object, i.e., a static obstacle. The second threshold may be smaller than the first threshold. If the brightness difference is smaller than the second threshold, the grid map generator 121 may set the smoothing coefficient k to a relatively large value, for example, k=0.99. If the brightness difference is between the first threshold and the second threshold, the grid map generator 121 may set the smoothing coefficient k to a predetermined value specified using a configuration file or the like.

[0035] The grid attribute determination unit 122 determines the grid attribute of each grid. Grid attributes include, for example, unknown, distorted, floor, and obstacle. For example, the grid attribute determination unit 122 determines the grid attribute of a grid in which no valid point cloud exists and the grid height indicates an invalid value as "unknown." The grid attribute determination unit 122 determines the grid attribute of a grid that is located at the edge of the imaging range and has significant distortion, such as a grid height that indicates a height deeper than the floor, as "distorted."

[0036] The grid attribute determination unit 122 determines the grid attribute of a grid whose grid height is equal to or greater than a predetermined height above the floor as "obstacle." Even if the grid attribute determination unit 122 has once determined an obstacle to be an "obstacle," if there are no other grids determined to be "obstacles" around the grid determined to be an "obstacle," the grid attribute determination unit 122 may consider the obstacle grid to be a false detection and correct the grid attribute from "obstacle" to another attribute, for example, "floor."

[0037] The grid attribute determination unit 122 may determine whether an obstacle is a dynamic obstacle or a static obstacle using the determination result of the grid attribute in a past frame. For example, when the determination result of the grid attribute changes from "floor" to "obstacle," the grid attribute determination unit 122 may determine the obstacle as a "dynamic obstacle" until a predetermined number of frames, for example, 40 frames, have elapsed since the frame in which the grid attribute changed. If the grid attribute determination unit 122 has continuously determined the obstacle to be an obstacle, it may determine the obstacle as a "static obstacle."

[0038] The grouping unit 123 groups obstacle grids, i.e., grids determined as "obstacles" by the grid attribute determination unit 122, into obstacle grid groups. The grouping unit 123 performs a labeling process on the obstacle grids and assigns the same label number to each group of obstacle grids. The grouping unit 123 groups obstacle grids that have been assigned the same label into one obstacle grid group. For example, a union find method may be used for grouping.

[0039] The obstacle detection unit 112 outputs a list of detected obstacles, i.e., a list of grouped obstacle grids, to the obstacle management unit 113. The list of obstacles detected by the obstacle detection unit 112 is also called a first obstacle list. The obstacle management unit 113 manages a list of obstacles that have already been detected. The list of detected obstacles is also called a second obstacle list. The list of detected obstacles is also called a second obstacle list. The obstacle identification numbers are managed based on the list of obstacles detected by the obstacle detection unit 112 and the list of obstacles that have already been detected.

[0040] In the following description, the list of obstacles detected by the obstacle detection unit 112 is also referred to as an obstacle detailed information list. Also, the list of detected obstacles managed by the obstacle management unit 113 is also referred to as an obstacle list. The obstacle list includes, for each obstacle identification number, a non-update count that indicates the time elapsed since the obstacle was last detected. The obstacle list also includes an update flag that indicates whether or not the information of an obstacle included in the obstacle list has been changed in each frame, i.e., whether or not the obstacle information has been updated. The update flag is initialized to non-update for each frame.

[0041] When an obstacle corresponding to an obstacle detected by the obstacle detection unit 112 is included in the obstacle list, the obstacle management unit 113 manages the detected obstacle by the identification number of the obstacle included in the obstacle list. The obstacle management unit 113 also updates the information of the obstacle included in the obstacle list with the information of the detected obstacle. Furthermore, the obstacle management unit sets the update flag to "updated."

[0042] On the other hand, if the obstacle list does not include an obstacle corresponding to the obstacle detected by the obstacle detection unit 112, the obstacle management unit 113 assigns a new identification number to the detected obstacle and manages the detected obstacle with the new identification number. The obstacle management unit 113 also adds information about the detected obstacle to the obstacle list as information about the new obstacle. The obstacle management unit 113 corresponds to the obstacle management unit 13 shown in FIG. 1.

[0043] FIG. 5 is a diagram showing a specific example of a list of obstacles detected by the obstacle detection unit 112. In the example of FIG. 5, the obstacle detailed information list includes an index, the number of grids, the minimum coordinate value, the maximum coordinate value, and an attribute. "Index" indicates the index of the obstacle. "Number of grids" indicates the number of grids in the obstacle grid group. "Minimum coordinate value" indicates the minimum value of the coordinates of the circumscribing rectangle that surrounds the obstacle grid group. "Maximum coordinate value" indicates the maximum value of the coordinates of the circumscribing rectangle that surrounds the obstacle grid group. "Attribute" indicates whether the obstacle grid group as a whole is a dynamic obstacle or a static obstacle.

[0044] FIG. 6 shows a list of detected obstacles managed by the obstacle management unit 113. In the example of FIG. 6, the obstacle list includes an element number, an identifier (ID), center coordinates, presence, non-update count, update flag, and index. "Element number" indicates the element number in the obstacle list. "ID" indicates the identification number of the obstacle. "Center coordinates" indicates the center coordinates of a circumscribing rectangle that surrounds the obstacle grid group.

[0045] "Existence" indicates whether an obstacle exists in the current frame. In other words, "existence" indicates whether an obstacle corresponding to an obstacle identified by an ID is included in the obstacle detail information list. "Number of non-updates" indicates the time that has elapsed since the obstacle was last detected. "Update flag" indicates whether the information about the obstacle included in the obstacle list has changed in each frame, i.e., whether the obstacle information has been updated. The update flag is initialized to False for each frame, indicating no update.

[0046] The obstacle management unit 113 compares the coordinate information of the obstacle included in the obstacle detailed information list with the coordinate information of the obstacle included in the obstacle list, and determines whether or not the obstacle list includes an obstacle corresponding to the obstacle detected by the obstacle detection unit 112. The obstacle management unit 113 sequentially selects the obstacles included in the obstacle detailed information list, i.e., the obstacles detected by the obstacle detection unit 112, one by one. The obstacle management unit 113 compares the coordinate information of the selected obstacle with the coordinate information of the obstacle included in the obstacle list, and determines whether or not the obstacle corresponding to the selected obstacle is included in the obstacle list.

[0047] The obstacle management unit 113 may sort the obstacle detailed information list and the obstacle list according to the coordinates of the obstacles. For example, the obstacle management unit 113 sorts the obstacle detailed information list and the obstacle list in ascending order of the x-coordinate and the y-coordinate, respectively, with the upper left corner of the image as the origin (0,0). The obstacle management unit 113 sorts the obstacles included in the obstacle detailed information list in ascending order, for example, based on the minimum coordinate value included in the obstacle detailed information list shown in FIG. 5. The obstacle management unit 113 also sorts the obstacles included in the obstacle detailed information list in ascending order, for example, based on the center coordinates included in the obstacle list shown in FIG. 6. The obstacle management unit 113 may select obstacles one by one sequentially from the top of the obstacle detailed information list sorted in ascending order. Furthermore, in the comparison, the coordinate information of the selected obstacle is compared with the coordinate information of the obstacle in the obstacle list sorted in ascending order. In this way, the order of comparison with the obstacle list can be fixed, and it is possible to prevent the identification numbers of adjacent obstacles from being swapped.

[0048] If an obstacle corresponding to the selected obstacle is included in the obstacle list, the obstacle management unit 113 identifies the obstacle corresponding to the selected obstacle. In other words, the obstacle management unit 113 identifies which obstacle included in the obstacle list the selected obstacle corresponds to. For example, the obstacle management unit 113 identifies an obstacle in the obstacle list whose center coordinates are closest to the center coordinates of the selected obstacle as the obstacle corresponding to the selected obstacle. The obstacle management unit 113 updates the information of the obstacle identified in the obstacle list based on the information of the selected obstacle.

[0049] For example, the obstacle management unit 113 overwrites the "center coordinates" of the identified obstacle in the obstacle list with values ​​corresponding to the coordinates of the obstacle selected from the obstacle detailed information list. Furthermore, the obstacle management unit 113 stores the index value of the obstacle selected from the obstacle detailed information list in the "index" of the identified obstacle in the obstacle list. In this way, the obstacle management unit 113 can manage the selected obstacle by the identification number of the obstacle that has already been detected. The obstacle management unit 113 sets the "update flag" of the identified obstacle to True, which indicates that the obstacle has been updated. Furthermore, the obstacle management unit 113 resets the "number of non-updates" to 0.

[0050] In the comparison, the obstacle management unit 113 may compare information about obstacles included in the obstacle list whose update flags are not set to "already updated" with information about the selected obstacle. This makes it possible to prevent information about an obstacle that has already been updated in the current frame in the obstacle list from being overwritten with information about another obstacle.

[0051] When all obstacles included in the obstacle detailed information list have been selected, the obstacle management unit 113 determines whether the obstacle list includes any obstacles that do not correspond to obstacles included in the obstacle detailed information list. In other words, the obstacle management unit 113 identifies obstacles that have not been detected in the current frame among the detected obstacles included in the obstacle list. The obstacle management unit 113 increments the "non-update count" of obstacles in the obstacle list that are determined not to correspond to obstacles included in the obstacle detailed information list.

[0052] The obstacle management unit 113 compares the non-update count of each obstacle included in the obstacle list with a predetermined threshold indicating a predetermined time. The predetermined threshold is defined, for example, as the number of processes per predetermined processing cycle. The obstacle management unit 113 identifies obstacles whose non-update count is greater than the predetermined threshold, i.e., obstacles whose non-update count indicates a time longer than the predetermined time. The obstacle management unit 113 deletes the identified obstacles from the obstacle list. For example, if the processing cycle is 100 ms and the non-update count threshold is 100, the obstacle management unit 113 deletes from the obstacle list any obstacle whose information has not been updated for 10 seconds. When the obstacle management unit 113 deletes an obstacle from the obstacle list, it deletes the obstacle information at the index associated with the deleted obstacle from the obstacle detailed information list.

[0053] The obstacle management unit 113 outputs the obstacle detail information list and the obstacle list to the obstacle information transmission unit 114. The obstacle information transmission unit 114 transmits the obstacle detail information list and the obstacle list to the mobile robot 150. The obstacle information transmission unit 114 may transmit information about obstacles detected in the current frame among the obstacles included in the obstacle list to the mobile robot 150. The control server 110 may generate movement path information for the mobile robot 150 based on the obstacle list acquired from the obstacle management unit 113, and transmit the generated movement path information to the mobile robot 150. The obstacle information transmission unit 114 corresponds to the control unit 14 shown in FIG. 1.

[0054] Next, the operation procedure will be explained. Fig. 7 is a flowchart showing the operation procedure of the control server 110. The operation procedure of the control server 110 corresponds to a control method. The depth information acquisition unit 111 acquires depth information of each location included in the movement range 200 of the mobile robot 150 based on image data of the movement range 200 acquired using the sensor 130 (step S1). In step S1, the depth information acquisition unit 111 acquires, for example, the height from the floor of each location as depth information.

[0055] The obstacle detection unit 112 detects obstacles present in the movement range 200 based on the depth information acquired in step S1 (step S2). In step S2, for example, the obstacle detection unit 112 generates a grid map obtained by dividing the movement range 200 into a plurality of grids based on the depth information. The obstacle detection unit 112 identifies, as obstacle grids, grids in the grid map whose depth information is equal to or greater than a predetermined height. Furthermore, the obstacle detection unit 112 groups the identified obstacle grids into one or more obstacle grid groups, and detects the grouped obstacle grid groups as obstacles.

[0056] The obstacle management unit 113 manages the identification numbers of obstacles based on the list of obstacles detected in step S2 and the list of previously detected obstacles (step S3). For example, in step S3, the obstacle management unit 113 determines whether an obstacle corresponding to the obstacle detected in step S2 is included in the list of previously detected obstacles. If it is determined that an obstacle corresponding to the detected obstacle is included in the list of previously detected obstacles, the obstacle management unit 113 assigns the identification number of the obstacle in the list of previously detected obstacles to the detected obstacle. Furthermore, the obstacle management unit 113 updates the information of the obstacle included in the list of previously detected obstacles with the information of the detected obstacle. If it is determined that an obstacle corresponding to the detected obstacle is not included in the list of previously detected obstacles, the obstacle management unit 113 assigns a new identification number to the detected obstacle. Furthermore, the obstacle management unit 113 adds the information of the detected obstacle to the list of previously detected obstacles as information of a new obstacle.

[0057] The list of detected obstacles includes, for each obstacle, a non-update count indicating the time elapsed since the obstacle was last detected. The obstacle management unit 113 deletes from the second obstacle list any obstacle whose non-update count indicates a time longer than a predetermined time (step S4). Steps S1 to S4 correspond to the operation procedure of the obstacle detection device 115. The operation procedure of the obstacle detection device 115 corresponds to an obstacle detection method.

[0058] The obstacle information transmitting unit 114 controls the mobile robot 150 by transmitting information about the obstacles included in the list of detected obstacles to the mobile robot 150 (step S5). The control server 110 repeatedly performs steps S1 to S5 at predetermined time intervals.

[0059] In this embodiment, the obstacle detection device 115 detects obstacles based on depth information acquired from a sensor 130, such as a stereo camera installed on the ceiling, in a warehouse environment where workers and the like are moving around. By using the information on the detected obstacles to control the mobile robot 150, it becomes possible for the mobile robot 150 to avoid obstacles on its travel path even when the obstacle is in the blind spot of the mobile robot 150, thereby improving safety and work efficiency.

[0060] In this embodiment, even if an obstacle is temporarily not detected, the obstacle management unit 113 holds information about the obstacle in the list of detected obstacles until the non-update count reaches a certain number. If an obstacle corresponding to an obstacle detected in the current frame is included in the list of detected obstacles, the obstacle management unit 113 manages the obstacle by the identification number of the obstacle included in the list of detected obstacles.

[0061] If the obstacle list only holds information about current or very recent obstacles, information about a temporarily disappeared obstacle is deleted from the obstacle list. When the obstacle reappears, it is detected as a new obstacle, and the obstacle's identification number changes. In contrast, if information about a disappeared obstacle is not deleted from the obstacle list, the temporarily disappeared obstacle can be managed with the same identification number as before. However, if information about an obstacle that has not been detected for a long time is not deleted from the obstacle list, the number of matching operations with obstacles detected in the current frame becomes enormous, resulting in a high processing load. Furthermore, there is a possibility that a new obstacle may be mistakenly associated with another obstacle that was detected in the past.

[0062] In this embodiment, information about a temporarily disappeared obstacle is maintained in the list of detected obstacles until the number of non-updates reaches a certain number, and when the number of non-updates reaches the certain number, the information about the disappeared obstacle is deleted from the list of detected obstacles. This prevents the number of matches with detected obstacles from becoming overwhelming, and even if an obstacle temporarily disappears, if the obstacle reappears, the reappeared obstacle can be managed with the same identification number as before. In this embodiment, the continuity of the obstacle identification numbers can be maintained. This has the advantage of making it easier to predict future obstacles compared to when identification numbers change frequently.

[0063] Next, a description will be given of the physical configuration of the control server 110 and the obstacle detection device 115. Fig. 8 is a block diagram showing an example configuration of a computer device that can be used as the control server 110 or the obstacle detection device 115. The computer device 500 has a processor 510 such as a CPU (Central Processing Unit), a storage unit 520, a ROM (Read Only Memory) 530, a RAM (Random Access Memory) 540, a communication interface (IF: Interface) 550, and a user interface 560.

[0064] The communication interface 550 is an interface for connecting the computer device 500 to a communication network via wired communication means or wireless communication means, etc. The user interface 560 includes a display unit such as a display, and an input unit such as a keyboard, a mouse, and a touch panel.

[0065] The storage unit 520 is an auxiliary storage device that can store various types of data. The storage unit 520 does not necessarily have to be a part of the computer device 500, but may be an external storage device or a cloud storage connected to the computer device 500 via a network.

[0066] The ROM 530 is a non-volatile storage device. For example, a semiconductor storage device with a relatively small capacity, such as a flash memory, is used for the ROM 530. The programs executed by the CPU 510 can be stored in the storage unit 520 or the ROM 530. The storage unit 520 or the ROM 530 stores various programs that realize the functions of each unit of the control server 110 or the obstacle detection device 115.

[0067] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include RAM, ROM, flash memory, solid-state drive (SSD) or other memory technology, compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0068] The RAM 540 is a volatile storage device. Various semiconductor memory devices such as a dynamic random access memory (DRAM) or a static random access memory (SRAM) are used for the RAM 540. The RAM 540 can be used as an internal buffer for temporarily storing data and the like. The CPU 510 loads a program stored in the storage unit 520 or the ROM 530 into the RAM 540 and executes it. The CPU 510 executes the program, thereby realizing the functions of each unit in the control server 110 or the obstacle detection device 115. The CPU 510 may have an internal buffer for temporarily storing data and the like.

[0069] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0070] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0071] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0072] [Appendix 1] a depth information acquisition unit that acquires depth information of a travel path based on sensor data from a sensor that senses the travel path of the mobile robot; an obstacle detection unit that detects one or more obstacles present on the travel path based on the depth information; an obstacle management unit that manages the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected; the second obstacle list includes, for each of the identification numbers, a non-update count indicating a time elapsed since the obstacle was last detected; The obstacle management unit deletes from the second obstacle list an obstacle for which the number of non-update times indicates a time period longer than a predetermined time.

[0073] [Appendix 2] the obstacle management unit, when an obstacle corresponding to the detected obstacle is included in the second obstacle list, assigns the detected obstacle an identification number of the obstacle included in the second obstacle list and updates information of the obstacle included in the second obstacle list with information of the detected obstacle; and, when an obstacle corresponding to the detected obstacle is not included in the second obstacle list, assigns a new identification number to the detected obstacle and adds information of the detected obstacle to the second obstacle list as information of a new obstacle.

[0074] [Appendix 3] 3. The obstacle detection device according to claim 2, wherein the obstacle management unit compares information about the obstacle included in the first obstacle list with information about the obstacle included in the second obstacle list, and determines whether an obstacle corresponding to the detected obstacle is included in the second obstacle list.

[0075] [Appendix 4] The obstacle detection device described in Appendix 3, wherein the obstacle management unit increments the non-update count for an obstacle included in the second obstacle list that is determined not to correspond to the one or more detected obstacles.

[0076] [Appendix 5] the second obstacle list further includes, for each obstacle, an update flag indicating whether information about the obstacle has been updated; The obstacle detection device described in Appendix 3 or 4, wherein the obstacle management unit sequentially selects obstacles included in the first obstacle list one by one, compares information about the selected obstacle with information about the obstacle included in the second obstacle list, and, if it determines that the selected obstacle is included in the second obstacle list, sets an update flag of the obstacle included in the second obstacle list and corresponding to the selected obstacle to updated.

[0077] [Appendix 6] The obstacle detection device described in Appendix 5, wherein the obstacle management unit compares information about obstacles included in the second obstacle list, the update flag of which is not set to updated, with information about the selected obstacle.

[0078] [Appendix 7] 7. The obstacle detection device according to claim 5, wherein the obstacle management unit sorts the obstacles included in the second obstacle list according to their coordinates on the travel path, and sequentially selects the obstacles from the first obstacle list in an order according to the coordinates of the obstacles.

[0079] [Appendix 8] the depth information indicates a height from a floor surface of the travel path, The obstacle detection device according to any one of appendixes 1 to 7, wherein the obstacle detection unit identifies, as obstacle grids, grids whose depth information is equal to or greater than a predetermined height in a grid map obtained by dividing the roadway into a plurality of grids, groups the identified obstacle grids into one or more obstacle grid groups, and detects the grouped obstacle grid groups as the obstacles.

[0080] [Appendix 9] 9. The obstacle detection device according to claim 1, wherein the sensor is installed on a ceiling of the travel path.

[0081] [Appendix 10] An obstacle detection device according to any one of appendices 1 to 9; a control unit that controls the mobile robot based on information about obstacles included in the second obstacle list.

[0082] [Appendix 11] acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; The obstacle detection method further comprises removing an obstacle from the second obstacle list for which the non-update count indicates a time period longer than a predetermined time period.

[0083] [Appendix 12] acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; removing an obstacle from the second obstacle list for which the non-update count indicates a time longer than a predetermined time; A control method for controlling the mobile robot based on information about obstacles included in the second obstacle list.

[0084] [Appendix 13] acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; A program that causes a computer to execute a process of deleting, from the second obstacle list, an obstacle for which the non-update count indicates a time period longer than a predetermined time period.

[0085] [Appendix 14] acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on information included in a first obstacle list including information about the detected obstacles and a second obstacle list including information about obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; removing an obstacle from the second obstacle list for which the non-update count indicates a time longer than a predetermined time; A program that causes a computer to execute a process including controlling the mobile robot based on information about obstacles included in the second obstacle list.

[0086] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 9 that are dependent on Supplementary Notes 1 may also be dependent on Supplementary Notes 11, 12, 13, and 14 in the same dependency relationship as Supplementary Notes 2 to 9. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0087] 10: Control device 11: Depth information acquisition section 12: Obstacle detection unit 13: Obstacle Management Department 14: Control unit 20: Obstacle detection device 100: System 110: Control server 111: Depth information acquisition unit 112: Obstacle detection unit 113: Obstacle Management Department 114: Obstacle information transmission unit 115: Obstacle detection device 121: Grid map generation unit 122: Grid attribute determination unit 123: Grouping section 130: Sensor 150: Mobile robot 200:Movement range

Claims

1. a depth information acquisition unit that acquires depth information of a travel path based on sensor data from a sensor that senses the travel path of the mobile robot; an obstacle detection unit that detects one or more obstacles present on the travel path based on the depth information; an obstacle management unit that manages the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected; the second obstacle list includes, for each of the identification numbers, a non-update count indicating a time elapsed since the obstacle was last detected; The obstacle management unit deletes from the second obstacle list an obstacle for which the number of non-update times indicates a time period longer than a predetermined time.

2. 2. The obstacle detection device according to claim 1, wherein, when an obstacle corresponding to the detected obstacle is included in the second obstacle list, the obstacle management unit assigns an identification number of the obstacle included in the second obstacle list to the detected obstacle and updates information of the obstacle included in the second obstacle list with the information of the detected obstacle, and when an obstacle corresponding to the detected obstacle is not included in the second obstacle list, the obstacle management unit assigns a new identification number to the detected obstacle and adds information of the detected obstacle to the second obstacle list as information of a new obstacle.

3. 3. The obstacle detection device according to claim 2, wherein the obstacle management unit compares information about the obstacles included in the first obstacle list with information about the obstacles included in the second obstacle list, determines whether an obstacle corresponding to the detected obstacle is included in the second obstacle list, and increments a non-update count for an obstacle included in the second obstacle list that is determined not to correspond to the one or more detected obstacles.

4. the second obstacle list further includes, for each obstacle, an update flag indicating whether information about the obstacle has been updated; 4. The obstacle detection device according to claim 3, wherein the obstacle management unit sequentially selects obstacles included in the first obstacle list one by one, compares information about the selected obstacle with information about the obstacle included in the second obstacle list, and, when determining that the selected obstacle is included in the second obstacle list, sets an update flag of the obstacle included in the second obstacle list and corresponding to the selected obstacle to "updated."

5. 5. The obstacle detection device according to claim 4, wherein the obstacle management unit sorts the obstacles included in the second obstacle list according to coordinates of the obstacles on the travel path, and sequentially selects the obstacles from the first obstacle list in an order according to the coordinates of the obstacles.

6. An obstacle detection device according to any one of claims 1 to 5; a control unit that controls the mobile robot based on information about obstacles included in the second obstacle list.

7. acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; The obstacle detection method further comprises removing an obstacle from the second obstacle list for which the non-update count indicates a time period longer than a predetermined time period.

8. acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; removing an obstacle from the second obstacle list for which the non-update count indicates a time longer than a predetermined time; A control method for controlling the mobile robot based on information about obstacles included in the second obstacle list.

9. acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on a first obstacle list including information on the detected obstacles and a second obstacle list including information on obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; a program that causes a computer to execute a process of deleting, from the second obstacle list, an obstacle for which the non-update count indicates a time period longer than a predetermined time period;

10. acquiring depth information of the travel path based on sensor data from a sensor that senses the travel path of the mobile robot; Detecting one or more obstacles present on the travel path based on the depth information; managing the identification numbers of the obstacles based on information included in a first obstacle list including information about the detected obstacles and a second obstacle list including information about obstacles that have already been detected, the second obstacle list including, for each identification number of the obstacle, a non-update count indicating the time elapsed since the obstacle was last detected; removing an obstacle from the second obstacle list for which the non-update count indicates a time longer than a predetermined time; a program for causing a computer to execute a process including controlling the mobile robot based on information about obstacles included in the second obstacle list;

Images