Method for operating a mobile robot, mobile robot, and object transport system

Abstract

The present invention relates to a working method of a mobile robot that carries articles in a manufacturing plant, a mobile robot, and an article carrying system including the mobile robot. According to an embodiment of the present invention, a working method of a mobile robot that carries articles in a manufacturing plant includes a step of obtaining a three-dimensional depth image; a step of extracting a region of interest from the three-dimensional depth image, which corresponds to a floor surface of the manufacturing plant and a travel path of the mobile robot; a step of generating a projected point cloud by projecting an object detected from the region of interest to a reference plane corresponding to the floor surface; a step of generating a virtual point cloud that fills a voxel in the reference plane; a step of comparing the virtual point cloud and the projected point cloud to detect a hole present in the floor surface; and a step of traveling while avoiding the hole.

Description

Technical Field

[0001] This invention relates to a method for operating a mobile robot, a mobile robot, and a goods transport system. Background Technology

[0002] Semiconductor or display manufacturing processes involve dozens or hundreds of steps on a substrate (wafer or glass) to produce the final product. Each process can be performed using manufacturing equipment that performs the corresponding process. If a process on a particular manufacturing device is completed, the item (substrate) can be moved to the next manufacturing device in order to proceed with the next process, and can be stored in storage equipment for a certain period of time.

[0003] As mentioned above, a goods handling system refers to a system for moving or storing goods in a manufacturing process. It can be broadly divided into a goods handling system and a goods storage system.

[0004] In material handling systems, in addition to overhead hoist transport (OHT) systems that travel along tracks set on the ceiling, mobile robots, similar to autonomous vehicles, have been introduced that move along the floor while simultaneously transporting materials. Mobile robots face the risk of collisions with various obstacles while navigating the interior spaces of manufacturing plants; therefore, various technologies using cameras, radar, or lidar are introduced to help them avoid these obstacles.

[0005] However, conventional obstacle detection technologies focus on methods for detecting obstacles located in the surrounding area. In the case of semiconductor manufacturing plants, where holes are formed on the bottom surface, methods are needed not only to detect obstacles but also to detect and avoid holes formed on the bottom surface. Summary of the Invention

[0006] Therefore, embodiments of the present invention provide a method for operating a mobile robot capable of detecting holes formed on the bottom surface of a manufacturing plant, a mobile robot, and an article transport system including the mobile robot.

[0007] The problems solved by the present invention are not limited to those mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art through the following description.

[0008] According to an embodiment of the present invention, a method for operating a mobile robot that transports items in a manufacturing plant includes: obtaining a three-dimensional depth image; extracting a region of interest from the three-dimensional depth image that corresponds to the travel path of the mobile robot on the ground of the manufacturing plant; projecting objects detected from the region of interest onto a reference plane corresponding to the ground to generate a projected point cloud; generating a virtual point cloud filled with voxels on the reference plane; comparing the virtual point cloud and the projected point cloud to detect holes present in the ground; and traveling around the holes.

[0009] According to an embodiment of the present invention, the step of extracting the region of interest may include: setting the region corresponding to the travel path of the mobile robot as the region of interest within the sensing range of the ground below the reference height in the three-dimensional depth image.

[0010] According to an embodiment of the present invention, the step of generating the projected point cloud may include: projecting voxels of the region of interest of the three-dimensional depth image onto the reference plane; and applying a voxel filter to the projected voxels to generate the projected point cloud.

[0011] According to an embodiment of the present invention, the voxel filter may reduce the number of voxels located on the reference plane.

[0012] According to an embodiment of the present invention, when the number of voxels located in a unit region in the reference plane is greater than or equal to a threshold value, the voxel filter transforms the voxels located in the unit region into a single voxel; when the number of voxels located in a unit region in the reference plane is less than the threshold value, the voxel filter deletes the voxels located in the unit region.

[0013] According to an embodiment of the present invention, the step of generating the virtual point cloud may include: generating a virtual point cloud in which a unit region of the voxel filter on the reference plane is filled with voxels.

[0014] According to an embodiment of the present invention, the step of detecting holes existing in the ground may include: removing voxels that overlap with the projected point cloud from the virtual point cloud to generate a remaining point cloud; and detecting the region in the reference plane where the voxels of the remaining point cloud exist as a hole region.

[0015] According to an embodiment of the present invention, the step of traveling to avoid the hole may include: applying a flip to the region of the voxels containing the remaining point cloud in the reference plane, thereby transforming the voxels of the hole region into obstacle objects in the three-dimensional depth image; and setting a movement path in a manner that avoids the obstacle objects and travels to the target location.

[0016] A mobile robot for transporting items in a manufacturing plant according to an embodiment of the present invention includes: a robot body that travels within the interior space of the manufacturing plant; a depth camera disposed on the robot body; and a processor that controls the movement of the robot body. The processor acquires a three-dimensional depth image from the depth camera, extracts a region of interest on the ground of the manufacturing plant corresponding to the movement path of the mobile robot from the three-dimensional depth image, projects objects detected from the region of interest onto a reference plane corresponding to the ground to generate a projected point cloud, generates a virtual point cloud filled with voxels on the reference plane, compares the virtual point cloud and the projected point cloud to detect holes existing in the ground, and controls the robot body to move in a manner that avoids the holes.

[0017] According to an embodiment of the present invention, the processor may define the area corresponding to the travel path of the mobile robot as the region of interest within the sensing range of the ground below the reference height in the three-dimensional depth image.

[0018] According to an embodiment of the present invention, the processor may project voxels of the region of interest of the three-dimensional depth image onto the reference plane and apply a voxel filter to the projected voxels to generate the projected point cloud.

[0019] According to an embodiment of the present invention, the voxel filter may reduce the number of voxels located on the reference plane.

[0020] According to an embodiment of the present invention, when the number of voxels located in a unit region in the reference plane is greater than or equal to a threshold value, the voxel filter transforms the voxels located in the unit region into a single voxel; when the number of voxels located in a unit region in the reference plane is less than the threshold value, the voxel filter deletes the voxels located in the unit region.

[0021] According to an embodiment of the present invention, the processor may generate a virtual point cloud in which a unit region of the voxel filter on the reference plane is filled with voxels.

[0022] According to an embodiment of the present invention, the processor may remove voxels that overlap with the projected point cloud from the virtual point cloud to generate a remaining point cloud, and detect the regions in the reference plane where voxels of the remaining point cloud exist as hole regions.

[0023] According to an embodiment of the present invention, the processor may apply a flip to the region of the voxels containing the remaining point cloud in the reference plane, thereby transforming the voxels of the hole region into obstacle objects in the three-dimensional depth image, and setting a movement path in a manner that avoids the obstacle objects and travels to the target position.

[0024] An article handling system in a manufacturing plant according to an embodiment of the present invention includes: a system controller for transmitting instructions for moving articles in the manufacturing plant; and a mobile robot for moving the articles according to the instructions. The mobile robot includes: a robot body for navigating within the interior space of the manufacturing plant; a depth camera disposed on the robot body to generate a three-dimensional depth image; and a processor for controlling the movement of the robot body, detecting obstacles in the robot's path in an upper region of interest of the three-dimensional depth image, and detecting holes in the floor of the manufacturing plant in a lower region of interest of the three-dimensional depth image. The processor generates a projected point cloud by projecting objects detected in the lower region of interest onto a reference plane corresponding to the floor, generating a virtual point cloud filled with voxels in the reference plane, and detecting holes in the floor by comparing the virtual point cloud and the projected point cloud, thereby controlling the robot body to avoid the holes.

[0025] According to an embodiment of the present invention, the mobile robot may transmit the position information of the hole to the system controller, and the system controller may store the position information of the hole and transmit it to other mobile robots.

[0026] According to an embodiment of the present invention, the processor may project voxels of the lower region of interest of the three-dimensional depth image onto the reference plane, generate the projected point cloud by applying a voxel filter to the projected voxels, generate a virtual point cloud in which the unit area of ​​the voxel filter in the reference plane is filled with voxels, generate a residual point cloud by removing voxels that overlap with the projected point cloud from the virtual point cloud, detect the area in the reference plane where the voxels of the residual point cloud exist as a hole region, and transform the voxels of the hole region in the three-dimensional depth image into obstacle objects by applying a flip to the area in the reference plane where the voxels of the residual point cloud exist, thereby setting the movement path of the robot body in a manner that avoids the obstacle objects and moves to the target position.

[0027] According to an embodiment of the present invention, when the number of voxels located in a unit region in the reference plane is greater than or equal to a threshold value, the voxel filter transforms the voxels located in the unit region into a single voxel; when the number of voxels located in a unit region in the reference plane is less than the threshold value, the voxel filter deletes the voxels located in the unit region.

[0028] According to the present invention, holes existing on the bottom surface of a manufacturing plant can be detected by comparing a point cloud projected from a three-dimensional depth image with a virtual point cloud.

[0029] The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art through the following description. Attached Figure Description

[0030] Figure 1 The invention provides a summary of manufacturing equipment in a manufacturing plant to which the invention can be applied, as well as a mobile robot for transporting goods.

[0031] Figure 2 as well as Figure 3 The outline configuration of the mobile robot according to the present invention is shown.

[0032] Figures 4 to 7 This is a flowchart illustrating the working method of the mobile robot according to the present invention.

[0033] Figures 8 to 21 This is an exemplary diagram illustrating the process of detecting holes present on the bottom surface from a three-dimensional depth image in an embodiment of the present invention.

[0034] Figure 22 The outline structure of the article transport system according to the present invention is shown.

[0035] Figure 23 This is a flowchart illustrating the working method of a mobile robot for detecting obstacles and holes in an article transport system according to the present invention.

[0036] Figure 24 The sensing range used to detect obstacles and holes is shown.

[0037] Figure 25 This is a signal flow diagram for an item transport system that uses shared obstacle location information.

[0038] (Explanation of reference numerals in the attached diagram)

[0039] 1: Manufacturing plant

[0040] 5: System Controller

[0041] 10: Mobile Robots

[0042] 110: Main body of the robot

[0043] 120: Depth Camera

[0044] 130: Processor Detailed Implementation

[0045] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art to which this invention pertains can readily implement it. The present invention can be implemented in various forms and is not limited to the embodiments described herein.

[0046] To clearly illustrate the present invention, parts unrelated to the description are omitted, and the same or similar constituent elements are given the same reference numerals throughout the specification.

[0047] Furthermore, in multiple embodiments, the same symbols are used for constituent elements having the same configuration, and the description is only given in the representative embodiment. In other embodiments, only configurations that differ from the representative embodiment are described.

[0048] In the context of the specification as a whole, when a part is "connected (or combined)" with other parts, this includes not only "direct connection (or combination)" but also "indirect connection (or combination)" by placing other components between them. Furthermore, when a part "includes" a constituent element, unless otherwise stated otherwise, it means that it does not exclude other constituent elements but rather includes them.

[0049] Unless otherwise defined, all terms used herein, including technical or scientific terms, shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms as defined in commonly used dictionaries shall be interpreted as having the same meaning in the context of the related art, and shall not be interpreted as having an idealized or overly formal meaning unless explicitly defined in this application.

[0050] Hereinafter, according to the present invention, a method for operating a mobile robot for moving items in a manufacturing plant, the mobile robot, and an item moving system including the mobile robot will be described.

[0051] Figure 1 The invention is illustrated in summary, showing the manufacturing equipment 2 of the manufacturing plant 1 to which the present invention can be applied, and the mobile robot 10 for transporting items.

[0052] like Figure 1As shown, manufacturing plant 1 is equipped with manufacturing equipment 2 for performing manufacturing processes, and a mobile robot 10 for transporting necessary items within the manufacturing equipment 2 moves within the internal space of manufacturing plant 1. For the production of semiconductor products, manufacturing equipment 2 can be equipment capable of performing various processes on wafers (e.g., coating, exposure, etching, vapor deposition, cleaning). Wafers can be transferred in a state of being stored in a FOUP (Front Opening Unified Pod) F. The mobile robot 10 can collect the FOUP F from a specific device and move to other devices while holding the FOUP F, and also transfer the FOUP F to other devices. Furthermore, in addition to FOUP F, the mobile robot 10 can also transport and store photomask pods used for exposure or other various materials.

[0053] Mobile robot 10, acting as a vehicle navigating the interior space of a manufacturing plant, can travel along either a designated path or an unspecified path. Mobile robot 10 can... Figure 22 The system controller 5 receives work instructions and moves according to them. Work instructions may include a starting point, a destination point, and item information. The mobile robot 10 can store map information of the manufacturing plant 1 in an internally configured memory (not shown), and generate a path from the starting point to the destination point from the map information. An exemplary configuration of the mobile robot 10 will be described below.

[0054] Figure 2 as well as Figure 3 The schematic configuration of a mobile robot 10 according to the present invention is shown. The mobile robot 10 for transporting items in a manufacturing plant 1 according to the present invention includes: a robot body 110 for traveling within the interior space of the manufacturing plant 1; a depth camera 120 disposed on the robot body 110; and a processor 130 for controlling the travel of the robot body 110.

[0055] To transport items, the robot body 110 is programmed to travel along a specific path within the manufacturing plant 1. The robot body 110 may include various drive devices for acceleration, deceleration, steering, and other actuation. Furthermore, to enable the robot body 110 to move, sensors and cameras for acquiring necessary information may be configured in the external or internal space of the robot body 110.

[0056] On the other hand, such as Figure 2 As shown, a storage section 115 for storing items on the upper part of the robot body 110 can be configured in the mobile robot 10. A storage area for storing items (e.g., FOUP F) can be configured within the storage section 115.

[0057] The depth camera 120 can capture images of the periphery of the robot body 110 to generate a 3D depth image, and then provide the 3D depth image to the processor 130. The depth camera 120 can be positioned outside the robot body 110, specifically in front, to determine the position of the robot body 110 or to identify obstacles in the travel path. Multiple depth cameras 120 can be configured; for example, one can be positioned in front and one in the rear, and multiple cameras can be positioned in front to capture images from different directions (left, right). The 3D depth image consists of multiple voxels in 3D space, each voxel defined by (x, y, z) coordinates.

[0058] The processor 130 may perform computational processing for the movement of the robot body 110 and control the drive mechanism of the robot body 110. The processor 130 may generally consist of multiple processing units (CPU, GPU) for computational processing and a controller for controlling the drive mechanism. Specifically, the processor 130 may detect not only obstacles but also holes (holes) existing in the bottom surface in a three-dimensional depth image captured by the depth camera 120, and control the robot body 110 to move in a manner that avoids obstacles and holes. The following describes the operation method of a mobile robot 10 capable of detecting holes formed in the bottom surface of the manufacturing plant 1; the operation method described below can be implemented by the processor 130.

[0059] Figures 4 to 7 This is a flowchart illustrating the working method of the mobile robot 10 according to the present invention.

[0060] The operating method of the mobile robot 10 according to the present invention includes: obtaining a three-dimensional depth image IMG1 (S410); extracting a region of interest (ROI) corresponding to the travel path of the mobile robot 10 from the three-dimensional depth image IMG1 on the bottom surface of the manufacturing plant 1 (S420); projecting objects detected from the region of interest (ROI) onto a reference plane RP corresponding to the bottom surface to generate a projected point cloud PC2 (S430); generating a virtual point cloud PC3 filled with voxels on the reference plane RP (S440); comparing the virtual point cloud PC3 and the projected point cloud PC2 to detect holes hl present on the bottom surface (S450); and traveling around the holes hl (S460).

[0061] In step (S410) of acquiring the 3D depth image IMG1, the following is obtained: Figure 8The image shown illustrates that while obstacles (ob) can be easily detected, holes (hl) formed on the bottom surface cannot be simply identified as obstacles because there are no corresponding voxel values. Embodiments of the present invention provide a method to identify holes (hl) formed on the bottom surface as obstacles and to allow the mobile robot 10 to avoid them during travel.

[0062] According to an embodiment of the present invention, the step of extracting the region of interest (ROI) (S420) may include setting the area corresponding to the travel path of the mobile robot 10 within the sensing range of the bottom surface centered on a reference height in the three-dimensional depth image IMG1 as the region of interest corresponding to the bottom surface of the manufacturing plant. For example, the existing height may be set as the height of the depth camera 120. Figure 9 This illustrates that the area corresponding to the travel path of the mobile robot 10 within the sensing range of the lower bottom surface is defined as the Region of Interest (ROI), such as... Figure 10 As shown, a 3D depth image of the region of interest (ROI) can be extracted. Figure 10 In the empty regions like those in the hologram, there are no voxels.

[0063] According to embodiments of the present invention, such as Figure 5 As shown, the step (S440) of generating the projected point cloud PC2 may include: projecting voxels of the region of interest (ROI) of the 3D depth image IMG1 onto the reference plane RP (S441); applying a voxel filter X to the projected voxel vx1 to generate the projected point cloud PC2 (S442).

[0064] In the step (S441) of projecting the voxels of the region of interest (ROI) onto the reference plane RP, Figure 10 A 3D depth image can be transformed into, for example... Figure 11 The point cloud PC1 of such a two-dimensional plane image. At this time, since there are no voxels in the region of hole hl, there are empty regions.

[0065] In order to Figure 11 The point cloud PC1 can be processed quickly and can be processed using voxel filters X.

[0066] According to an embodiment of the present invention, the voxel filter X is a filter that reduces the number of voxels vx1 located in the reference plane RP. When the number of voxels located in a unit region ua in the reference plane RP is greater than or equal to a threshold value, the voxel filter X transforms the voxels located in the unit region ua into a voxel vx2. When the number of voxels located in the unit region ua in the reference plane RP is less than the threshold value, the voxel filter X deletes the voxels vx1 located in the unit region ua. For example, the voxel filter X can be configured as follows: Figure 12The dimensions (width, height) of the voxel filter X are the same as the dimensions of the region of interest (ROI). For example... Figure 13 As shown, if the voxel filter X is applied to the point cloud PC1, the number of voxels is calculated in each unit region ua. Within a specific unit region ua, when the number of voxels exceeds a critical value, the voxel vx1 of the corresponding unit region ua is transformed as follows: Figure 14 A voxel vx2. Within a specific unit region ua, when the number of voxels does not reach a critical value, such as... Figure 14 In that case, the voxel vx1 of the corresponding unit region ua is deleted and the corresponding unit region ua becomes empty.

[0067] Therefore, as Figure 14 In this way, a point cloud PC2 is generated and projected onto the reference plane RP. The projected point cloud PC2 includes simplified voxels vx2.

[0068] Thus, by using a voxel filter X, the amount of data and computing resources required for detection holes can be reduced, and data can be processed quickly.

[0069] According to an embodiment of the present invention, the step (S440) of generating a virtual point cloud PC3 includes generating a virtual point cloud PC3 in which a unit region ua of the voxel filter X on the reference plane RP is filled with voxels vx3. Figure 15 As shown, virtual point clouds PC3 can be generated with the same size as the region of interest (ROI) and each unit region ua of the voxel filter X can be filled with voxels vx3. The generation of virtual point clouds PC3 is used to separate the hole regions in the ROI.

[0070] According to an embodiment of the present invention, the step (S450) of detecting a hole hl on the bottom surface may include: removing voxels that overlap with point cloud PC2 in the virtual point cloud PC3 to generate a remaining point cloud PC4 (S451), and detecting the area in the reference plane RP where the voxels of the remaining point cloud PC4 exist as a hole area (S452).

[0071] like Figure 16 As shown, in the virtual point cloud PC3, the voxels that overlap with the projected point cloud PC2 generate the removed residual point cloud PC4. The residual point cloud PC4 includes the residual voxels vx4 in the virtual point cloud PC3 that do not overlap with the projected point cloud PC2. The region containing the residual voxels vx4 can be detected as a hole region.

[0072] According to an embodiment of the present invention, the step of avoiding the hole (S460) may include: applying a flip to the region of voxel vx4 in the reference plane RP where there is a remaining point cloud PC4, thereby transforming the voxel of the hole region into an obstacle object ob_hl in the three-dimensional depth image IMG1 (S461); and setting a movement path in a manner that avoids the obstacle object ob_hl and moves toward the target position (S462).

[0073] right Figure 16 The remaining point cloud PC4 is suitable for flipping, such as Figure 17 In this way, a 3D point cloud filled with voxels (vx4) is generated in the hole region. The voxels (vx4) in the hole region are then transformed into obstacle objects (ob_hl), thus enabling the generation of objects such as... Figure 18 The transformation of the 3D depth image IMG2. The mobile robot 10 can avoid the obstacle object ob_hl in the 3D depth image IMG2 while driving. In conclusion, even if a hole is formed on the bottom surface, the corresponding hole can also be identified as an obstacle and avoided.

[0074] Figure 19 An example of a 3D depth image captured by a depth camera 120 is shown. If a region of interest is extracted from the 3D depth image, a result such as... can be generated. Figure 20 Such an image, and through the previously described comparison processing steps of the projected point cloud and the virtual point cloud, can be used to extract... Figure 21 This includes point clouds of voxels formed in the hole region. By flipping the corresponding voxels, the mobile robot 10 can avoid the holes formed on the bottom surface, just like obstacles.

[0075] The following describes the working method of the mobile robot 10, which is capable of detecting holes formed on the bottom surface of the manufacturing plant 1, implemented by the processor 130.

[0076] According to the present invention, the processor 130 of the mobile robot 10 obtains a three-dimensional depth image IMG1 from the depth camera 120, extracts the region of interest (ROI) corresponding to the driving path of the mobile robot 10 on the bottom surface of the manufacturing plant 1 from the three-dimensional depth image IMG1, and projects the objects detected from the region of interest (ROI) onto the reference plane RP corresponding to the bottom surface, thereby generating a projected point cloud PC2, generating a virtual point cloud PC3 filled with voxels on the reference plane RP, comparing the virtual point cloud PC3 and the projected point cloud PC2 to detect holes hl existing on the bottom surface, and controlling the robot body 110 to drive in a manner that avoids holes hl.

[0077] According to an embodiment of the present invention, the processor 130 sets the area corresponding to the travel path of the mobile robot 10 as the region of interest (ROI) corresponding to the bottom surface of the manufacturing plant 1 within the sensing range of the three-dimensional depth image IMG1 centered on the reference height.

[0078] According to an embodiment of the present invention, the processor 130 may project voxels of the region of interest (ROI) of the three-dimensional depth image IMG1 onto the reference plane RP, and apply a voxel filter X to the projected voxels vx1 to generate the projected point cloud PC2.

[0079] According to an embodiment of the present invention, the voxel filter X is a filter that reduces the number of voxels vx1 located in the reference plane RP. When the number of voxels located in a unit region ua in the reference plane RP is greater than or equal to a threshold value, the voxel filter X transforms the voxels located in the unit region ua into a voxel vx2. When the number of voxels located in the unit region ua in the reference plane RP is less than the threshold value, the voxel filter X deletes the voxels vx1 located in the unit region ua.

[0080] According to an embodiment of the present invention, the processor 130 can generate a virtual point cloud PC3 in which a unit region ua of the voxel filter X on the reference plane RP is filled with voxel vx3.

[0081] According to an embodiment of the present invention, the processor 130 can generate a residual point cloud PC4 by removing voxels that overlap with the projected point cloud PC2 from the virtual point cloud PC3, and detect the region where the voxels of the residual point cloud PC4 exist in the reference plane RP as a hole region.

[0082] According to an embodiment of the present invention, the processor 130 can apply flipping to the region of voxel vx4 in the reference plane RP where there is residual point cloud PC4, thereby transforming the voxel of the hole region into obstacle object ob_hl in the three-dimensional depth image IMG1, and setting a movement path in a manner that avoids the obstacle object ob_hl and moves toward the target position.

[0083] Figure 22 The outline structure of the article transport system according to the present invention is shown.

[0084] According to the present invention, in a manufacturing plant 1, the goods transport system includes: a system controller 5, which transmits instructions for transporting goods in the manufacturing plant 1; and a mobile robot 10, which transports goods according to the instructions.

[0085] When items need to be transferred within manufacturing plant 1, system controller 5 generates instructions for item transfer and transmits these instructions to the mobile robot 10. System controller 5 receives item transfer instructions from the control system described above. System controller 5 can control not only the mobile robot 10 but also other types of transfer devices (e.g., OHT (Overhead Hoist Transport), OHS (Overhead Hoist Shuttle), RGV (Rail Guided Vehicle)). System controller 5 and mobile robot 10 can transmit and receive signals wirelessly. Although not shown, mobile robot 10 may include a communication module for wireless communication, through which processor 130 can transmit or receive signals.

[0086] The mobile robot 10 can be configured to move according to instructions received from the system controller 5, and the mobile robot 10 can be configured to avoid obstacles and holes located on the movement path.

[0087] According to an embodiment of the present invention, the mobile robot 10 designates regions of interest for detecting obstacles and regions of interest for detecting holes in the captured three-dimensional depth image, and performs obstacle and hole detection simultaneously (in parallel). The mobile robot 10 according to this embodiment will now be described.

[0088] According to an embodiment of the present invention, the mobile robot 10 includes: a robot body 110 that travels within the interior space of a manufacturing plant 1; a depth camera 120 disposed on the robot body 110 to generate a three-dimensional depth image; and a processor 130 that controls the travel of the robot body 110, detects obstacles located on the travel path of the mobile robot in the upper region of interest of the three-dimensional depth image, and detects holes located on the bottom surface of the manufacturing plant in the lower region of interest of the three-dimensional depth image.

[0089] Alternatively, the processor 130 may project the object detected from the lower region of interest onto a reference plane RP corresponding to the bottom surface, thereby generating a projected point cloud PC2, generating a virtual point cloud PC3 filled with voxels on the reference plane RP, comparing the virtual point cloud PC3 and the projected point cloud PC2 to detect the hole hl present on the bottom surface, and controlling the robot body 110 to move in a manner that avoids the hole hl.

[0090] Figure 23 This is a flowchart of the working method of a mobile robot 10 for detecting obstacles and holes in an article transport system according to the present invention.

[0091] This can be achieved by the mobile robot 10 acquiring a 3D depth image IMG1 via the depth camera 120 (S2310), detecting obstacles located on the travel path of the mobile robot 10 in the upper region of interest of the 3D depth image IMG1 (S2320), and detecting holes located on the bottom surface of the manufacturing plant 1 in the lower region of interest of the 3D depth image IMG1 (S2325). Figure 24 As shown, the upper region is set as the upper obstacle sensing range from the reference height, and the lower region is set as the lower bottom surface sensing range from the reference height. The upper area of ​​interest is set within the upper obstacle sensing range to correspond to the travel path of the mobile robot 10, and the lower area of ​​interest is set within the lower obstacle sensing range to correspond to the travel path of the mobile robot 10.

[0092] On the other hand, the location information of the hole detected by the mobile robot 10 can be shared with other mobile robots, and obstacle information can be shared to prevent collisions with obstacles or falling into the hole in advance. Here, not only the location information of the hole can be shared, but also information about the shape of the hole (size, appearance).

[0093] According to an embodiment of the present invention, the mobile robot 10 may transmit the hole position information to the system controller 5, and the system controller 5 may store the hole position information and transmit it to other mobile robots.

[0094] For example, such as Figure 25 As shown, the system controller 5 may transmit a driving command to the first mobile robot 10-1 (S2402), and detect obstacles or holes during the process of the first mobile robot 10-1 moving towards the target location (S2404) (S2406). After the first mobile robot 10-1 avoids the obstacle or hole and moves, it transmits the detection information of the obstacle or hole to the system controller 5 (S2408). The detection information may include the location information of the obstacle or hole, as well as the size and shape of the obstacle or hole. Alternatively, the system controller 5 may store the obstacle or hole information (S2410) and share the corresponding information with other mobile robots. In particular, the system controller 5 may transmit the corresponding obstacle or hole information (S2414) while transmitting a driving command (S2412) to the second mobile robot 10-2, which is expected to move along a similar path around the obstacle or hole. The second mobile robot 10-2 may then consider the obstacle or hole information (position, shape) and move towards the target location (S2416). For example, in the presence of obstacles or holes, the second mobile robot 10-2 can travel at a relatively low speed, and if no obstacles or holes are detected, it can also transmit a notification message that the corresponding obstacle or hole has been removed to the system controller 5.

[0095] This embodiment and the accompanying drawings only explicitly illustrate a part of the technical concept included in the present invention. Within the scope of the technical concept included in the specification and drawings of the present invention, all variations and specific embodiments that can be easily deduced by those skilled in the art should be included within the scope of the claims of the present invention.

[0096] Therefore, the concept of the present invention is not limited to the described embodiments, and all equivalents or variations thereof not only to the appended claims should fall within the scope of the concept of the present invention.

Claims

1. A method for operating a mobile robot to transport items in a manufacturing plant, wherein, The working method of the mobile robot includes: Steps to obtain a 3D depth image; The step of extracting the region of interest on the ground of the manufacturing plant corresponding to the travel path of the mobile robot from the three-dimensional depth image; The step of projecting objects detected from the region of interest onto a reference plane corresponding to the ground to generate a projected point cloud; The step of generating a virtual point cloud that fills the reference plane with voxels; The step of detecting holes existing in the ground by comparing the virtual point cloud and the projected point cloud; and The steps to avoid the hole.

2. The method for operating a mobile robot according to claim 1, wherein, The steps for extracting the region of interest include: The step of defining the area corresponding to the travel path of the mobile robot as the region of interest within the sensing range of the ground below the reference height in the three-dimensional depth image.

3. The method for operating a mobile robot according to claim 1, wherein, The steps for generating the projected point cloud include: The steps of projecting the voxels of the region of interest in the three-dimensional depth image onto the reference plane; and The step of applying a voxel filter to the projected voxels to generate the projected point cloud.

4. The method for operating a mobile robot according to claim 3, wherein, The voxel filter reduces the number of voxels located in the reference plane.

5. The method for operating a mobile robot according to claim 4, wherein, When the number of voxels located in a unit region in the reference plane is greater than or equal to a threshold value, the voxel filter transforms the voxels located in the unit region into a single voxel. When the number of voxels located in a unit region in the reference plane is less than a critical value, the voxel filter deletes voxels located in the unit region.

6. The method for operating a mobile robot according to claim 5, wherein, The steps for generating the virtual point cloud include: The step of generating a virtual point cloud in which a unit region of the voxel filter on the reference plane is filled with voxels.

7. The method for operating a mobile robot according to claim 6, wherein, The steps for detecting holes present in the ground include: The step of removing voxels that overlap with the projected point cloud from the virtual point cloud to generate the remaining point cloud; and The step of detecting regions in the reference plane where voxels of the remaining point cloud exist as hole regions.

8. The method for operating a mobile robot according to claim 7, wherein, The steps for driving around the hole include: The step of applying a flip to the region of voxels containing the remaining point cloud in the reference plane, thereby transforming the voxels of the aperture region into obstacle objects in the three-dimensional depth image; and The step of setting a movement path in a manner that avoids the obstacle object and moves toward the target location.

9. A mobile robot for transporting goods in a manufacturing plant, wherein, The mobile robot includes: The robot body moves within the interior space of the manufacturing plant; A depth camera is mounted on the robot's main body; and The processor controls the movement of the robot's main body. The processor obtains a three-dimensional depth image from the depth camera, extracts a region of interest (ROI) on the ground of the manufacturing plant corresponding to the travel path of the mobile robot from the three-dimensional depth image, projects objects detected from the ROI onto a reference plane corresponding to the ground to generate a projected point cloud, generates a virtual point cloud filled with voxels on the reference plane, compares the virtual point cloud with the projected point cloud to detect holes existing in the ground, and controls the robot body to travel in a manner that avoids the holes.

10. The mobile robot according to claim 9, wherein, The processor defines the area corresponding to the mobile robot's travel path as the region of interest within the sensing range of the ground below the reference height in the three-dimensional depth image.

11. The mobile robot according to claim 9, wherein, The processor projects voxels of the region of interest in the 3D depth image onto the reference plane and applies a voxel filter to the projected voxels to generate the projected point cloud.

12. The mobile robot according to claim 11, wherein, The voxel filter reduces the number of voxels located in the reference plane.

13. The mobile robot according to claim 12, wherein, When the number of voxels located in a unit region in the reference plane is greater than or equal to a threshold value, the voxel filter transforms the voxels located in the unit region into a single voxel. When the number of voxels located in a unit region in the reference plane is less than a critical value, the voxel filter deletes voxels located in the unit region.

14. The mobile robot according to claim 13, wherein, The processor generates a virtual point cloud that fills a unit region of the voxel filter on the reference plane with voxels.

15. The mobile robot according to claim 14, wherein, The processor removes voxels that overlap with the projected point cloud from the virtual point cloud to generate a residual point cloud, and detects the regions in the reference plane where voxels of the residual point cloud exist as hole regions.

16. The mobile robot according to claim 15, wherein, The processor applies a flip to the region of voxels containing the remaining point cloud in the reference plane, thereby transforming the voxels of the hole region into obstacle objects in the three-dimensional depth image, and setting a movement path in a manner that avoids the obstacle objects and travels to the target location.

17. A goods transport system for transporting goods in a manufacturing plant, wherein, The goods transport system includes: The system controller transmits instructions for moving items within the manufacturing plant; and The mobile robot transports the items according to the instructions. The mobile robot includes: The robot body moves within the interior space of the manufacturing plant; A depth camera is installed on the robot body to generate a three-dimensional depth image; The processor controls the movement of the robot body, detects obstacles in the robot's path in the upper region of interest of the 3D depth image, and detects holes in the ground of the manufacturing plant in the lower region of interest of the 3D depth image. The processor generates a projected point cloud by projecting objects detected from the lower region of interest onto a reference plane corresponding to the ground, generates a virtual point cloud that fills the reference plane with voxels, and detects holes existing in the ground by comparing the virtual point cloud and the projected point cloud, thereby controlling the robot body to move in a manner that avoids the holes.

18. The goods conveying system according to claim 17, wherein, The mobile robot transmits the location information of the hole to the system controller. The system controller stores the location information of the hole and transmits it to other mobile robots.

19. The goods conveying system according to claim 17, wherein, The processor projects voxels of the lower region of interest in the 3D depth image onto the reference plane, generates a projected point cloud by applying a voxel filter to the projected voxels, generates a virtual point cloud in which a unit area of ​​the voxel filter in the reference plane is filled with voxels, generates a residual point cloud by removing voxels that overlap with the projected point cloud from the virtual point cloud, detects the region in the reference plane where the voxels of the residual point cloud exist as a hole region, and transforms the voxels of the hole region in the 3D depth image into an obstacle object by applying a flip to the region in the reference plane where the voxels of the residual point cloud exist, thereby setting the movement path of the robot body in a manner that avoids the obstacle object and moves to the target position.

20. The goods conveying system according to claim 19, wherein, When the number of voxels located in a unit region in the reference plane is greater than or equal to a threshold value, the voxel filter transforms the voxels located in the unit region into a single voxel. When the number of voxels located in a unit region in the reference plane is less than a critical value, the voxel filter deletes voxels located in the unit region.

Images