Article storage area detection device, article transfer device, article storage area detection method, and article transfer method

Abstract

To provide an article storage area detection device capable of shortening a processing time required for detecting a storage area for storing an article in a container.SOLUTION: The distance image acquisition unit 44 acquires a distance image of the inside of the container 13. The extraction unit 47 extracts information on the storage space in the container and information on the stored objects existing in the container from the acquired distance image. A virtual area arranging part 50 arranges a virtual area including articles to be stored in a storage space in a vertical direction and a horizontal direction. Interference determination unit 50 determines the interference with the storage item with respect to the virtual region from the virtual region of the upper layer among the virtual regions disposed in the storage space, performs the interference determination of the virtual region of the next lower layer with which the virtual region without interference overlaps in a case where it is determined that there is no interference, and does not perform the interference determination with respect to the virtual region of the lower layer with which the virtual region with interference overlaps in a case where it is determined that there is interference.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an article storage area detection device that detects a storage area for storing articles in a container, an article transfer device that uses this article storage area detection device, an article storage area detection method, and an article transfer method. [Background technology]

[0002] BACKGROUND ART As described in Patent Document 1, for example, a technique is known in the art for determining a storage area in a container for storing an item based on shape data of the item and the container when storing the item in the container. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-208604 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-described technology, there is a demand for reducing the time required to detect a storage area in a container where an article is stored.

[0005] The problem that the present invention aims to solve is to provide an item storage area detection device, an item transfer device, an item storage area detection method, and an item transfer method that can shorten the processing time required to detect a storage area for storing items in a container. [Means for solving the problem]

[0006] The item storage area detection device of the present invention is an item storage area detection device that detects a storage area for storing items within a container, and includes: a distance image acquisition unit that acquires a distance image within the container; an extraction unit that extracts information about the storage space within the container and information about the stored items present in the container from the acquired distance image; a virtual area placement unit that places a virtual area containing the items to be stored in the storage space in the vertical and horizontal directions; and an interference detection unit that determines interference between the stored items and the virtual area from an upper virtual area among the virtual areas placed within the storage space, and if it determines that there is no interference, performs interference detection for the virtual area one layer below with which the virtual area without interference overlaps, and if it determines that there is interference, does not perform interference detection for the virtual area in the layer below with which the virtual area with interference overlaps. [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce the processing time required to detect a storage area for storing an item in a container. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a side view of an article transfer device equipped with an article storage area detection device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a plan view of a container that stores articles using the article transfer device. [Figure 3] 10 is a perspective view showing the articles to be stored in a container by the article transfer device and a virtual area that contains the articles. FIG. [Figure 4] FIG. 2 is a block diagram of the article transfer device. [Figure 5] 10A and 10B show an example of arranging a virtual area in the storage space inside a container using the same item storage area detection device, where (a) is an explanatory diagram showing the arrangement of the virtual area in the reference position, and (b) is an explanatory diagram showing the arrangement of the virtual area in the rotated position. [Figure 6] 4 is a flowchart showing the operation of detecting a storage area by the item storage area detection device. [Figure 7]4 is a flowchart showing details of an interference determination operation performed by the item storage area detection device. [Figure 8] 5A to 5D are explanatory diagrams showing the procedure of an interference determination operation performed by the item storage area detection device. [Figure 9] FIG. 10 is a block diagram of an article transfer device equipped with an article storage area detection device according to a second embodiment of the present invention. [Figure 10] 1A and 1B show virtual areas generated by the item storage area detection device, where FIG. 1A is an explanatory diagram of one virtual area, and FIG. 1B is an explanatory diagram of virtual areas arranged in the storage space inside a container. [Figure 11] 4 is a flowchart showing the operation of detecting a storage area by the item storage area detection device. [Figure 12] 4 is a flowchart showing details of a storage area detection operation performed by the item storage area detection device. [Figure 13] 1A and 1B are diagrams explaining the detection operation of the item storage area detection device, in which (a) is a plan view of the virtual area, (b) is an explanatory diagram of the height distribution within the virtual area, (c) is a plan view showing the coordinates of the abandoned position in the virtual area, and (d) is a side view showing the coordinates of the abandoned position in the virtual area. DETAILED DESCRIPTION OF THE INVENTION

[0009] A first embodiment of the present invention will be described below with reference to FIGS.

[0010] 1 shows an article transfer device 10. The article transfer device 10 picks up an article 12 stored in a container 11 as a transfer source and transfers it into another container 13 as a transfer destination.

[0011] The article transfer device 10 is equipped with a transfer machine 15 that picks an article 12 from inside a container 11 and transfers it into a container 13. The container 11 is transported by a conveyor 16 to a picking position for the transfer machine 15, and the container 13 is transported by a conveyor 17 to a transfer position for the transfer machine 15. Above the picking position, a measurement unit 19 is installed that measures the inside of the container 11 at the picking position from above, and above the transfer position, a measurement unit 20 is installed that measures the inside of the container 13 at the transfer position from above.

[0012] The transfer machine 15 includes a hand device 22 that grips the article 12 and a movement mechanism 23 that moves the hand device 22. The hand device 22 includes a suction pad that suctions the top surface of the article 12, palms that clamp the sides of the article 12, and the like.

[0013] The movement mechanism 23 is, for example, an articulated robot, and includes a robot arm 25 rotatably mounted on a base 24. The robot arm 25 has a rotation shaft 26 at its tip, and the hand device 22 is attached to the lower end of this rotation shaft 26, allowing the hand device 22 to be moved to any position. The robot arm 25 is controlled so that the central axis of the rotation shaft 26 remains vertical, thereby making it possible to adjust the horizontal rotation angle of the hand device 22 around the rotation shaft 26. Note that the movement mechanism 23 is not limited to an articulated robot, and may be, for example, an orthogonal movement mechanism configured with three orthogonal slide axes.

[0014] As shown in Fig. 2, the container 13 is a box-shaped container, for example, having a bottom 29 and four side surfaces 30 rising from the periphery of the bottom 29, with the top surface open. A cubic storage space 31 is formed inside the container 13. The container 13 is formed in a rectangular shape with long side surfaces 30 and short side surfaces 30 in a plan view. Note that the container 11 also has an open top surface, similar to the container 13, allowing the transfer machine 15 to pick up the item 12.

[0015] 2 and 3, the article 12 comes in various shapes and sizes, such as a box-like shape or a cylindrical shape. Fig. 2 shows a state in which the article 12 is stored in the container 13, and the article 12 and the like in the container 13 become the stored object (obstructing object) 32 stored in the container 13.

[0016] Next, a block diagram of the article transfer device 10 is shown in FIG.

[0017] The item transfer device 10 includes an item storage area detection device 40 that detects a storage area for storing items 12 in a container 13, a transfer machine 15, and a transfer machine control device (robot controller) 41 that controls the transfer machine 15.

[0018] The item storage area detection device 40 includes an item information acquisition unit 43 , a distance image acquisition unit 44 , and a control unit 45 .

[0019] The item information acquisition unit 43 acquires information about the shape and dimensions of the item 12 from the measurement unit 19 installed above the picking position. The measurement unit 19 uses an RGBD camera or the like having a sensor capable of acquiring distance information (depth information) such as a three-dimensional (3D) image and a sensor capable of acquiring RGB information, and for example, the inside of the container 11 onto which inspection light of a random dot pattern is projected is photographed from different angles using two cameras, a distance image including the inside of the container 11 is acquired based on triangulation, and information about the shape and dimensions of the item 12, coordinates which are its position within the container 11, and posture is acquired by image processing the distance image. In addition, if a host machine such as a server has information about the shape and dimensions of the item 12 to be picked, the item information acquisition unit 43 may acquire the information about the shape and dimensions of the item 12 from the host machine.

[0020] The distance image acquisition unit 44 includes a measurement unit 20 installed above the transfer position, and acquires a distance image including the inside of the container 13 at the transfer position. The distance image acquisition unit 44 uses a three-dimensional sensor (3D sensor) or a three-dimensional camera (3D camera), and for example, uses two cameras to capture images from different angles of the inside of the container 13 onto which inspection light of a random dot pattern is projected, and acquires a distance image including the inside of the container 13 based on triangulation.

[0021] The control unit 45 includes an extraction unit 47 , a virtual area generation unit 48 , a virtual area placement unit 49 , an interference determination unit 50 , and a storage area selection unit 51 .

[0022] The extraction unit 47 performs image processing on the distance image acquired by the distance image acquisition unit 44 to extract information on the shape, dimensions, position, and posture (orientation) of the storage space 31 within the container 13, as well as information on the shape, dimensions, position, and posture of the stored items 32 present within the container 13.

[0023] The virtual area generation unit 48 generates information about a rectangular parallelepiped virtual area 53 (see FIG. 3) that encompasses the item 12 based on information about the shape and dimensions of the item 12 from the measurement unit 19. Note that a measurement unit that measures the item 12 while the hand device 22 is holding it may be installed separately from the measurement units 19 and 20, and the virtual area 53 may be generated based on the dimensional information of the item 12 acquired by the measurement unit, or registered data for the virtual area 53 that corresponds to the data of the item 12 may be called up and used.

[0024] The virtual area placement unit 49 places the virtual area 53 in the vertical and horizontal directions within the extracted storage space 31. In this case, the virtual area placement unit 49 places the virtual area 53 in the storage space 31 in a reference posture, which is a first posture in which the long side direction of the storage space 31 and the long side direction of the virtual area 53 coincide, as shown in Fig. 5(a), and a rotated posture, which is a second posture in which the long side direction of the virtual area 53 is perpendicular to the longitudinal direction of the storage space 31, as shown in Fig. 5(b).

[0025] The interference detection unit 50 determines whether an upper virtual region 53 among the virtual regions 53 arranged in the storage space 31 interferes with the stored item 32. If it determines that there is no interference (the virtual region 53 is empty), it performs interference detection for the virtual region 53 in the layer below that overlaps with the non-interfering virtual region 53. If it determines that there is interference, it does not perform interference detection for the virtual region 53 in the layer below that overlaps with the interfering virtual region 53. Specifically, the interference detection unit 50 starts interference detection from the virtual region 53 at the reference position A (see FIGS. 5(a) and 5(b)) of the topmost layer of the virtual regions 53 arranged in the storage space 31. If it determines that there is no interference, it performs interference detection for the virtual region 53 in the layer below. If it determines that there is interference, it performs interference detection for the virtual region 53 in the same layer for which interference detection has not been performed. At this time, the interference detection unit 50 performs interference detection for both the reference attitude and the rotated attitude of the virtual region 53. Note that the top layer may be a layer where a specific layer is set as the top layer.

[0026] The storage area selection unit 51 selects the virtual area 53 on the lower layer between the virtual area 53 determined to be free of interference in the reference posture and the virtual area 53 determined to be free of interference in the rotated posture as the storage area for the item 12 in the container 13. Furthermore, if the lower layers of the virtual area 53 determined to be free of interference in the reference posture and the virtual area 53 determined to be free of interference in the rotated posture are the same layer, the storage area selection unit 51 selects the virtual area 53 determined to be free of interference in the reference posture as the storage area for storing the item 12 in the container 13.

[0027] In addition, the transfer machine control device 41 obtains information regarding the shape, dimensions, coordinates, and posture of the item 12 stored in the container 11 from the measurement unit 19, as well as information regarding the coordinates and posture of the storage area detected by the item storage area detection device 40, and controls the transfer machine 15 to remove the item 12 from the container 11 at the picking position and transfer it to the detected storage area in the container 13 at the transfer position.

[0028] Next, the operation of detecting a storage area for storing the article 12 in the container 13 by the article storage area detection device 40 will be described with reference to the flowcharts of FIGS. 6 and 7 and the explanatory diagrams of FIGS.

[0029] It is determined whether the container 13 placed at the transfer position can be measured (step S1). The hand device 22 of the transfer machine 15 moves from the area above the container 13, and measurement of the container 13 becomes possible when the distance image acquisition unit 44 (measurement unit 20) is able to take an image of the inside of the container 13.

[0030] If it is possible to measure container 13 (YES in step S1), distance image acquisition unit 44 (measurement unit 20) captures a distance image including the interior of container 13, thereby measuring the interior of container 13 (step S2). This process is called the distance image acquisition step. Extraction unit 47 processes the distance image acquired by distance image acquisition unit 44 to extract information on the shape, dimensions, position, and orientation of storage space 31 in container 13, and information on the shape of stored items 32 present in container 13. This process is called the extraction step.

[0031] It is determined whether it is possible to measure the item 12 in the container 11 placed at the picking position (step S3). The hand device 22 of the transfer machine 15 moves from the area above the container 11, and the measurement unit 19 can photograph the inside of the container 11, and the item 12 can be measured.

[0032] If it is possible to measure the item 12 (YES in step S3), the measurement unit 19 takes a distance image including the inside of the container 11, thereby measuring the dimensions of the item 12 inside the container 11 (step S4). The distance image taken by the measurement unit 19 is subjected to image processing to extract information about the shape and dimensions of the item 12, and the information about the shape and dimensions of the item 12 is acquired by the item information acquisition unit 43.

[0033] Based on the information about the shape and dimensions of the item 12 acquired by the item information acquisition unit 43, the virtual area generation unit 48 generates information about a rectangular parallelepiped virtual area 53 (see FIG. 3) that encompasses the item 12 (step S5). At this time, information about both the virtual area 53 in the reference posture and the rotated posture is generated. This process is called the virtual area generation step. Here, the virtual area 53 may be created based on dimensional information acquired from the placed item 12 or the item 12 held by the hand device 22, or registered data for the virtual area 53 corresponding to the data for the item 12 may be called and used.

[0034] The virtual area placement unit 49 places the virtual areas 53 at predetermined intervals in the vertical and horizontal directions within the storage space 31 extracted by the extraction unit 47. At this time, a reference posture model is generated that places the virtual areas 53 in the storage space 31 in a reference posture in which the long side direction of the storage space 31 and the long side direction of the virtual areas 53 coincide, as shown in Fig. 5(a), and a rotated posture model is generated that places the virtual areas 53 in the storage space 31 in a rotated posture in which the long side direction of the virtual areas 53 is perpendicular to the longitudinal direction of the storage space 31, as shown in Fig. 5(b) (step S6). This process is referred to as the virtual area placement step.

[0035] 5(a) shows an example in which a virtual area 53 at one corner of the top layer is set as the reference position A, and three virtual areas 53 are arranged in a stacking direction a from the reference position A toward the bottom of the storage space 31, four virtual areas 53 are arranged in a vertical direction (first horizontal arrangement direction) b from the reference position A toward the lengthwise direction in the horizontal direction of the storage space 31, and three virtual areas 53 are arranged in a horizontal direction (second horizontal arrangement direction) c from the reference position A toward the widthwise direction in the horizontal direction of the storage space 31. The virtual areas 53 adjacent in the vertical and horizontal directions may be arranged so that they partially overlap or may be arranged apart. In the indexes (a, b, c) in the stacking direction a, vertical direction b, and horizontal direction c relative to the reference position A, the reference position A is set as index (0, 0, 0), and each virtual area 53 is assigned an index such that the stacking direction a is indexed (0, 0, 0), (1, 0, 0), and (2, 0, 0), the horizontal direction c is indexed (0, 0, 0), (0, 0, 1), and (0, 0, 2), and the vertical direction b is indexed (0, 0, 0), (0, 1, 0), (0, 2, 0), and (0, 3, 0), thereby setting the determination order in the layer. Furthermore, when the virtual area 53 to be determined is moved to a lower layer, collision determination is performed on the virtual area 53 in the next determination order of the layer to which the virtual area 53 to be determined belongs.

[0036] 5(b) shows an example in which a virtual area 53 at one corner of the top layer is set as a reference position A, and three virtual areas 53 are arranged in a stacking direction a from the reference position A toward the bottom of the storage space 31, five virtual areas 53 are arranged in a vertical direction b from the reference position A toward the lengthwise direction in the horizontal direction of the storage space 31, and two virtual areas 53 are arranged in a horizontal direction c from the reference position A toward the widthwise direction in the horizontal direction of the storage space 31. The virtual areas 53 adjacent in the vertical and horizontal directions may be arranged so that they partially overlap or may be arranged apart. With reference position A as the base, the indexes (a, b, c) are assigned to the stacking direction a, vertical direction b, and horizontal direction c. Reference position A is set as index (0, 0, 0), and each virtual area 53 is assigned an index as follows: index (0, 0, 0), (1, 0, 0), (2, 0, 0) in the stacking direction a; index (0, 0, 0), (0, 1, 0), (0, 2, 0), (0, 3, 0), (0, 4, 0) in the vertical direction b; and index (0, 0, 0), (0, 0, 1) in the horizontal direction c.

[0037] The virtual regions 53 overlapping in the stacking direction a are called stack rows, the virtual regions 53 aligned in the vertical direction b are called vertical rows, and the virtual regions 53 aligned in the horizontal direction c are called horizontal rows.

[0038] Then, the interference determination unit 50 determines whether or not there is interference with the stored item 32 in the virtual area 53 arranged in the storage space 31, from the top to the bottom. At this time, interference determination is performed for both the reference posture and the rotated posture of the virtual area 53 (step S7). This process is called the interference determination step.

[0039] Here, the details of the interference determination by the interference determination unit 50 in step S7 will be described with reference to the sub-process flowchart of FIG. 7 and FIGS. 5(a) and 5(b).

[0040] An initial value of index (a, b, c) is set as reference position A. In the flowchart, virtual area 53 with index (a, b, c) = (0, 0, 0) is set as the initial value (step S21). The virtual area 53 with the set index (a, b, c) is used as the object of interference detection to determine whether it interferes with the stored item 32 (step S22).

[0041] As a result of the determination, if it is determined that there is no interference (no interference in step S22), the determined virtual area 53 is registered as an interference-free area (index (a, b, c)) (step S23).

[0042] Thereafter, it is determined whether the index (a) in the stacking direction a of the virtual area 53 determined to have no interference is maximum (a=MAX), that is, whether the bottom layer has been reached (step S24). If it is not maximum (NO in step S24), the value of the index a is updated to (a+1) (step S25), and interference determination is performed on the virtual area 53 in the layer immediately below the virtual area 53 determined to have no interference (step S26).

[0043] In this way, interference determination is performed in order from the upper layer to the lower layer in the stacking sequence of virtual areas 53 overlapping in the stacking direction a, as long as it is determined that there is no interference. If the determination result of the virtual area 53 of the lowest layer also indicates that there is no interference, and it is determined in step S24 that the index (a) is the largest (YES in step S24), the virtual area 53 of the lowest layer determined to be an interference-free area is determined as a storage area (index (a, b, c)) (step S27), and information that there is a storage area is output (step S28).

[0044] Furthermore, if it is determined in step S22 or step S26 that interference has occurred (interference in step S22 or interference in step S26), it is determined whether the index (c) in the horizontal direction c for the virtual region 53 determined to have interfered is maximum (c=MAX), that is, whether it has reached the most downstream end in the horizontal direction c (the rightmost end in FIGS. 5(a) and 5(b)) relative to the reference position A (step S29). If it is not maximum (NO in step S29), the value of the index c is updated to (c+1) (step S30). Thereafter, the process returns to step S22, and interference determination is performed for the virtual region 53 that is adjacent in the horizontal direction c to the stacked layer sequence that has interfered and is in the same layer as the virtual region 53 determined to have interfered. For example, in Figure 5(a) or (b), if it is determined that there is interference in the virtual area 53 with index (1,0,0) in the second layer from the top at reference position A, then an interference determination is made for the virtual area 53 with index (1,0,1) in the second layer from the top in the second stacked row adjacent in the horizontal direction c.

[0045] If it is determined in step S22 that there is no interference, as described above, the process proceeds to step S23, where it is registered as no interference, and then it is determined in step S24 whether it is the bottom layer.If it is not the bottom layer, the process proceeds to step S25, where it checks for interference in the virtual area 53 of the layer below, and if it is the bottom layer, the process proceeds to step S27, where it is confirmed as a storage area.

[0046] Furthermore, if it is determined in step S29 that the index (c) of the virtual region 53 in the horizontal direction c is maximum (c=MAX), the index is reset to (c=0) (step S31), and it is determined whether the index (b) in the vertical direction b is maximum (b=MAX), that is, whether the most downstream end in the vertical direction b relative to the reference position A (the deepest side in FIGS. 5(a) and 5(b)) has been reached (step S32). If it is not maximum (NO in step S32), the value of the index b is updated to (b+1) (step S33). Thereafter, the process returns to step S22, and interference determination is performed for the virtual region 53 in the stacked row located in the vertical direction b relative to the reference position A and in the same layer as the virtual region 53 determined to have interfered. For example, in Figure 5(a), if it is determined that there is interference in the virtual area 53 with index (1,0,2) in the second layer from the top at the most upstream end in the vertical direction b (the foremost side in Figure 5(a)) and the most downstream end in the horizontal direction c (the rightmost side in Figure 5(a)), then an interference determination is made for the virtual area 53 with index (1,1,0) in the second layer from the top (the layer one below index (0,1,0)) in the stacked row that is the second row in the vertical direction b and the most upstream end in the horizontal direction c (the leftmost side in Figure 5).

[0047] Furthermore, if it is determined in step S32 that the index (b) in the vertical direction b of virtual area 53 is maximum (b=MAX) (YES in step S32), it is determined whether an interference-free area is registered (step S34), and if an interference-free area is registered (YES in step S34), the process proceeds to step S27, where the virtual area 53 registered as an interference-free area is confirmed as a storage area (index (a, b, c)), and information that a storage area exists is output. Furthermore, if an interference-free area is not registered (NO in step S34), information that there is no storage area for storing item 12 in container 13 is output (step S35).

[0048] Such interference determination of the virtual area 53 is performed for both the reference posture and the rotated posture of the virtual area 53 .

[0049] 8(a) to 8(d) show an example of the operation of detecting interference between a series of virtual regions 53. In Fig. 8(a) to 8(d), non-interfering virtual regions 53 are marked with T, and interfering virtual regions 53 are marked with F.

[0050] An interference check is performed for the virtual region 53 with index (0,0,0) at the reference position A (FIG. 8(a)). If interference is determined to exist, an interference check is performed for the virtual region 53 with index (0,0,1) in the top layer of the stacked row adjacent to the virtual region 53 at the reference position A in the horizontal direction c (FIG. 8(b)). If it is determined that there is no interference for the virtual region 53 with index (0,0,1), an interference check is performed for the virtual region 53 in the next lower layer (index (1,0,1)) (FIG. 8(c)). If it is determined that there is interference for the virtual region 53 with index (1,0,1), an interference check is performed for the virtual region 53 with index (1,0,2) in the second layer of the stacked row adjacent to the virtual region 53 with index (1,0,2) in the horizontal direction c (FIG. 8(d)).

[0051] 6, if a storage area is found based on the output result of the interference determination in step S7 (YES in step S8), it is checked whether there are multiple storage areas (step S9), and if there are multiple storage areas (YES in step S9), one storage area is selected by the storage area selection unit 51 (step S10). The storage area selection unit 51 selects the virtual area 53 with the lowest vertical position as the storage area for storing the item 12 in the container 13 between the virtual area 53 determined to be free of interference in the reference posture and the virtual area 53 determined to be free of interference in the rotated posture. Furthermore, if the lowest vertical positions of the virtual area 53 determined to be free of interference in the reference posture and the virtual area 53 determined to be free of interference in the rotated posture are the same, the storage area selection unit 51 selects the virtual area 53 determined to be free of interference in the reference posture as the storage area. This process is referred to as a storage area selection step.

[0052] If it is determined in step S8 that there is no storage area (NO in step S8), the storage area detection process is terminated. In this case, the item 12 is not stored in the container 13 that has no storage area, and an alternative corresponding container 13 is prepared, and the item 12 is stored in that container 13.

[0053] If there is one storage area in step S9 (NO in step S9), or after selecting one storage area in step S10, the coordinates, posture information, etc. of that storage area are output to the transfer machine control device 41 (step S11), and if there is another item 12 to be transferred (YES in step S12), the process returns to step S1, and if there is no another item 12 to be transferred (NO in step S12), the process ends.

[0054] Then, the transfer machine control device 41, which acquires information about the storage area from the item storage area detection device 40, controls the transfer machine 15 to transfer the item 12 taken out from the container 11 at the picking position to the detected storage area within the container 13 at the transfer position.

[0055] As described above, the item storage area detection device 40 determines whether or not there is interference with the stored items 32 among the virtual areas 53 arranged in the storage space 31, from the top to the bottom, and performs interference detection for the virtual area 53 in the layer below that overlaps with the virtual area 53 determined to have no interference.However, it does not perform interference detection for the virtual area 53 in the layer below that overlaps with the virtual area 53 determined to have interference.Therefore, the number of virtual areas 53 for which interference detection is performed can be reduced, and the processing time required to detect the storage area for storing the items 12 in the container 13 can be shortened.

[0056] Furthermore, among the virtual areas arranged within the storage space 31, the determination of interference with the stored items 32 begins with the virtual area 53 at the reference position of the top layer, and if it is determined that there is no interference, an interference determination is made for the virtual area 53 in the layer below.If it is determined that there is interference, an interference determination is made for the virtual area 53 in the same layer using another group of virtual areas 53 that overlap in the vertical direction and for which interference determination has not been made.This minimizes the number of virtual areas 53 for which interference determination is made, and further reduces the processing time required to detect storage areas.

[0057] In addition, the virtual area 53 is placed in the storage space 31 and interference is determined for a reference posture in which the long side direction of the storage space 31 and the long side direction of the virtual area 53 are aligned, and a rotated posture in which the long side direction of the virtual area 53 is perpendicular to the longitudinal direction of the storage space 31.Then, of the virtual area 53 determined to have no interference in the reference posture and the virtual area 53 determined to have no interference in the rotated posture, the virtual area 53 with the lowest vertical position is selected as the storage area for storing the item 12 in the container 13.Furthermore, if the virtual area 53 determined to have no interference in the reference posture and the virtual area 53 determined to have no interference in the rotated posture have the same lowest vertical position, the virtual area 53 determined to have no interference in the reference posture is selected as the storage area for storing the item 12 in the container 13, thereby improving the storage efficiency of the item 12 in the container 13.

[0058] Next, a second embodiment is shown in Figures 9 to 13. Note that the same components as those in the first embodiment are denoted by the same reference numerals and their description will be omitted.

[0059] FIG. 9 shows a block diagram of the article transfer device 10.

[0060] The control unit 45 of the item storage area detection device 40 includes an extraction unit 47 , a virtual area generation unit 48 , a virtual area placement unit 49 , a storage area detection unit 60 , and a discharge position setting unit 61 .

[0061] Based on information relating to the shape and dimensions of the item 12 from the measurement unit 19 and information relating to the storage space 31 of the container 13 from the distance image acquisition unit 44 (measurement unit 20), the virtual area generation unit 48 generates information about a virtual area 53 (see FIG. 10(a)) whose horizontal dimension is the dimension including the item 12 and whose vertical dimension is the dimension (H+α) including the height dimension H within the container 13. Note that a measurement unit that measures the item 12 while the hand device 22 is holding it may be provided separately from the measurement units 19 and 20, and the virtual area 53 may be generated based on the dimensional information of the item 12 acquired by the measurement unit, or registered data for the virtual area 53 corresponding to the data of the item 12 may be called up and used.

[0062] The virtual area placement unit 49 places the virtual area 53 in the horizontal direction (horizontal direction c, vertical direction b) within the extracted storage space 31 (see FIG. 10(b)). At this time, the virtual area placement unit 49 places the virtual area 53 in the storage space 31 for both the reference posture and the rotated posture.

[0063] The storage area detection unit 60 extracts the highest point based on the height of the bottom of the storage space 31 or the height of the stored items 32 for each virtual area 53 placed within the storage space 31, and detects the virtual area 53 with the lowest highest point as the storage area.

[0064] The discharge position setting unit 61 sets the discharge position for discharging the article 12 into the storage area as the height position at which the lower surface height of the article 12 is the highest point within the storage area.

[0065] The operation of detecting a storage area in which the article 12 is stored in the container 13 by the article storage area detection device 40 will be described with reference to the flowcharts of FIGS.

[0066] In FIG. 11, steps S41 to S44 are the same as steps S1 to S4 described above.

[0067] In step S45, based on information about the shape and dimensions of the item 12 acquired by the item information acquisition unit 43 and information about the storage space 31 of the container 13 acquired by the distance image acquisition unit 44, information about a virtual area 53 (see Figure 10(a)) is generated whose horizontal dimension is a dimension that encompasses the item 12 and whose vertical dimension is a dimension that includes the height dimension within the container 13 as the height of the storage area. At this time, information about both the virtual area 53 in the reference posture and the rotated posture is generated. Note that this process is called the virtual area generation step. Furthermore, the height of the storage area is a height that allows the container 13 to store the item 12 and can be selected arbitrarily.

[0068] The virtual area placement unit 49 places the virtual areas 53 generated by the virtual area generation unit 48 in the storage space 31 extracted by the extraction unit 47, lining them up at predetermined intervals in the horizontal direction (horizontal direction c, vertical direction b) (see FIG. 10(b)). At this time, a reference posture model shown in FIG. 10(b) and a rotated posture model (not shown) are generated (step S46). This process is called the virtual area placement step.

[0069] Then, the storage area detection unit 60 performs a detection process to extract the bottom of the storage space 31 or the highest point of the height of the stored item 32 for each virtual area 53 arranged within the storage space 31, and detects the virtual area 53 with the lowest highest point as the storage area. At this time, the detection process is performed for both the reference attitude and the rotated attitude of the virtual area 53 (step S47). This process is called the storage area detection step.

[0070] Details of the detection process by the storage area detection unit 60 in step S47 will now be described with reference to the sub-process flowchart of FIG. 12 and FIGS. 13(a), (b), and (c).

[0071] For each virtual area 53 arranged within the storage space 31, the highest point based on the bottom of the storage space 31 or the height of the stored item 32 is extracted (step S61). Figure 13(a) shows an example in which the highest points h1, h2, h3, and h4 are extracted for four adjacent virtual areas 53.

[0072] The virtual area 53 in which the extracted highest point is at the lowest position is extracted as a candidate for the storage area (each candidate for the reference posture and the rotated posture) (step S62). If the highest points of the four virtual areas 53 shown in Figure 13(a) have the relationship h2>h3>h4>h1, the virtual area 53 in which the lowest highest point h1 exists is detected as a candidate for the storage area.

[0073] The highest point within the storage area is set at the center of the detected storage area (step S63), and this is output as the release position when the transfer machine 15 releases the item 12 into the container 13 (step S64). The release position is the position of the center of the bottom surface of the item 12 when the transfer machine 15 releases the item 12 into the container 13. For example, while the height distribution within the storage area is extracted as shown in FIG. 13(b), the release position p is set at the center of the storage area and at the same height as the highest point h1 within the storage area. In this case, for example, as shown in the plan view of the virtual area 53 in FIG. 13(c) and the side view of the virtual area 53 in FIG. 13(d), the coordinates of the release position p are set within the storage area (shown by diagonal lines). This process is called the release position setting step. In step S64, information on the coordinates of the set release position p, information on whether the position is the reference posture or the rotated posture, etc. are output.

[0074] Such detection of the storage area and setting of the release position are performed for both the reference attitude and the rotational attitude of the virtual area 53.

[0075] Then, returning to the flowchart of FIG. 11, based on the result of the storage area detection in step S47, the storage area and release position of the orientation in which the highest point or release position is lowest are selected from the storage areas of the reference orientation and the rotation orientation (step S48).

[0076] It is determined whether the transfer machine 15 can release the item 12 into the container 13 from the release position of the selected storage area (step S49). If the transfer machine 15 interferes with the container 13 or the item 12 protrudes beyond the container 13 by more than a predetermined height, it is determined that the item 12 cannot be stored in the container 13 (NO in step S49).

[0077] If the transfer machine 15 is able to store the item 12 in the container 13 (YES in step S49), information on the coordinates of the release position in the storage area, information on whether it is in the reference position or the rotated position, etc. is output to the transfer machine control device 41 (step S50), and if there is an item 12 to be transferred next (YES in step S51), the process returns to step S41, and if there is no item 12 to be transferred next (NO in step S51), the process ends.

[0078] Then, the transfer machine control device 41, which has acquired information regarding the storage area from the item storage area detection device 40, controls the transfer machine 15 to transfer the item 12 removed from the container 11 at the picking position to the detected storage area in the container 13 at the transfer position so that the center of the underside of the item 12 is positioned at the release position, and releases the item 12 into the container 13 by releasing the hand device from its grip on the item 12.

[0079] As described above, in the item storage area detection device 40, virtual areas 53 are arranged horizontally within the storage space 31 of the container 13, with horizontal dimensions that encompass the items 12 and vertical dimensions that include the height dimension within the container 13. For each virtual area 53 arranged within the storage space 31, the highest point of the height of the bottom of the storage space 31 or the stored items 32 is extracted, and the virtual area 53 with the lowest highest point is detected as the storage area.As a result, fewer virtual areas 53 are required for detection processing, and the processing time required to detect the storage area that stores the items 12 within the container 13 can be shortened.

[0080] By setting the release position for releasing the article 12 in the storage area as a height position where the height of the lower surface of the article 12 is the highest point in the storage area, the article 12 moved to the storage area for transfer of the article 12 can be released without pressing the stored articles 32 stored in the storage area from above.

[0081] In the virtual area placement steps S6 and S46, the virtual areas 53 are not only placed so that they do not overlap with each other, but also placed with a predetermined shift amount so that the virtual areas 53 partially overlap with each other.

[0082] Although the embodiment of the present invention and its modified examples have been described above, various combinations of configurations, partial omissions, substitutions and modifications are also possible. [Explanation of symbols]

[0083] 10 Article transfer device 12 Goods 13 Container 15 Transfer machine 22 Hand device 31 Storage space 32 Storage items 40 Item storage area detection device 44 Range image acquisition unit 47 Extraction part 49 Virtual Area Placement Unit 50 Interference detection section 51 Storage area selection section 53 Virtual Realm 60 Storage area detection unit 61 Release position setting section A Reference position p release position

Claims

1. An article storage area detection device that detects a storage area for storing articles in a container, a distance image acquisition unit that acquires a distance image of the inside of the container; an extraction unit that extracts information about the storage space in the container and information about the items stored in the container from the acquired distance image; a virtual area arrangement unit that arranges a virtual area containing the item to be stored in the storage space in the vertical and horizontal directions; an interference determination unit that determines interference between the stored item and an upper virtual area among the virtual areas arranged in the storage space, and when determining that there is no interference, performs interference determination for the virtual area in the next lower layer with which the virtual area without interference overlaps, and when determining that there is interference, does not perform interference determination for the virtual area in the next lower layer with which the virtual area with interference overlaps; An item storage area detection device comprising:

2. The interference determination unit starts interference determination from one of a plurality of virtual region groups in which the virtual regions arranged in the storage space overlap in the vertical direction, the virtual region being at a reference position of the top layer, and when determining that there is no interference, performs interference determination for the virtual region in the layer one layer below in the same virtual region group, and when determining that there is interference, performs interference determination for the virtual region in the same layer in the virtual region group for which interference determination has not been performed.

2. The article storage area detection device according to claim 1.

3. the virtual area placement unit places the virtual area in the storage space with respect to a reference orientation in which a long side direction of the storage space and a long side direction of the virtual area coincide with each other and a rotated orientation in which the longitudinal direction of the storage space and the long side direction of the virtual area are orthogonal to each other; The interference determination unit performs interference determination for the virtual region for both the reference posture and the rotational posture.

2. The article storage area detection device according to claim 1.

4. a storage area selection unit that selects, as the storage area, the virtual area in a lower layer between the virtual area determined to be free of interference in the reference posture and the virtual area determined to be free of interference in the rotated posture; 4. The article storage area detection device according to claim 3.

5. When the virtual area determined to have no interference in the reference posture and the virtual area determined to have no interference in the rotated posture are on the same lower layer, the storage area selection unit selects the virtual area determined to have no interference in the reference posture as the storage area.

5. The article storage area detection device according to claim 4.

6. An article storage area detection device that detects a storage area for storing articles in a container, a distance image acquisition unit that acquires a distance image of the inside of the container; an extraction unit that extracts information about the storage space in the container and information about the items stored in the container from the acquired distance image; a virtual area arrangement unit that arranges a virtual area horizontally in the storage space, the virtual area having a horizontal dimension sufficient to enclose the item and a vertical dimension including a height dimension of the storage area; a storage area detection unit that extracts the bottom of the storage space or the highest point of the height of the stored items for each of the virtual areas arranged within the storage space, and detects the virtual area with the lowest highest point as the storage area; An item storage area detection device comprising:

7. a release position setting unit that sets a release position for releasing the article in the storage area so that the height of the lower surface of the article is at the height position that is the highest point in the storage area; 7. The article storage area detecting device according to claim 6.

8. The item storage area detection device according to any one of claims 1 to 7, a transfer machine having a hand device that grasps the article, acquiring information about the storage area detected by the article storage area detection device, and transferring the article grasped by the hand device to the storage area in the container based on the information about the storage area; An article transfer device comprising:

9. An article storage area detection method for detecting a storage area for storing articles in a container, comprising: a distance image acquisition step of acquiring a distance image of the inside of the container; an extraction step of extracting information about the storage space in the container and information about items stored in the container from the acquired distance image; a virtual area arrangement step of arranging a virtual area in the vertical direction and horizontal direction within the storage space, the virtual area including the items to be stored; an interference determination step of determining interference between the stored item and an upper virtual area among the virtual areas arranged in the storage space, and if it is determined that there is no interference, performing interference determination for the virtual area in the layer below that overlaps with the virtual area without interference, and if it is determined that there is interference, not performing interference determination for the virtual area in the layer below that overlaps with the virtual area with interference; An item storage area detection method comprising:

10. The interference determination step performs interference determination from the virtual area of ​​an upper layer arranged in the storage space to the virtual area of ​​a lower layer, and when it is determined that there is no interference with the virtual area to be determined, performs interference determination for the virtual area of ​​the layer one below the virtual area to be determined, and when it is determined that there is interference with the virtual area to be determined, performs interference determination for the virtual area in the same layer as the virtual area to be determined and for which interference determination has not been performed. The item storage area detecting method according to claim 9 .

11. the interference determination step performs interference determination in the order from the upper layer of the virtual area to the lower layer of the virtual area arranged in the storage space, and in the order of determination set for the layers of the virtual areas; If it is determined that there is no interference with the virtual region to be determined, the virtual region one layer below the virtual region to be determined is used as the object of determination for interference detection, and if it is determined that there is interference with the virtual region to be determined, the interference detection is performed for the virtual region in the same layer as the virtual region to be determined and in the next determination order. The item storage area detecting method according to claim 9 .

12. An article storage area detection method for detecting a storage area for storing articles in a container, comprising: a distance image acquisition step of acquiring a distance image of the inside of the container; an extraction step of extracting information about the storage space in the container and information about items stored in the container from the acquired distance image; a virtual area arrangement step of horizontally arranging the virtual area in the storage space, the virtual area having a horizontal dimension sufficient to enclose the item and a vertical dimension including a height dimension of the storage area; a storage area detection step of extracting the bottom of the storage space or the highest point of the height of the stored items for each of the virtual areas arranged in the storage space, and detecting the virtual area with the lowest highest point as the storage area; An item storage area detection method comprising:

13. a discharge position setting step for setting a discharge position for discharging the article in the storage area so that the height of the lower surface of the article is at the height position that is the highest point in the storage area; The item storage area detection method according to claim 12 .

14. A transfer machine that acquires information about the storage area detected by the article storage area detection method according to any one of claims 9 to 13 transfers the article held by a hand device of the transfer machine to the storage area in the container. An article transfer method characterized by:

Images