Loading automation support device, loading automation support method, and program

Abstract

To assist loading automation carried out by a transportation system.SOLUTION: A loading automation assistance device 10 comprises: a lateral data acquisition unit 11 that acquires three-dimensional point group data generated from output data of a depth sensor installed on the lateral side of a mobile body; a load detection unit 12 that detects the surface of the mobile body front side of a load disposed on a load carrying platform of the mobile body by using three-dimensional point group data of a set height or more among the acquired three-dimensional point group data; a tool detection unit 13 that specifies point group data located on the uppermost surface of a tool disposed on the load carrying platform by using, among the acquired three-dimensional point group data, three-dimensional point group data located on the mobile body front side of the surface of the load, and detects the surface of the mobile body rear side of the tool by using the specified point group data; and a distance calculation unit 14 that calculates a distance between the surface of the load and the surface of the tool.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a loading automation support device and a loading automation support method that support loading in a luggage transportation system, and further to a program for implementing these. [Background technology]

[0002] Generally, cranes are used in factories to transport heavy loads, such as coils of thin plates, machine parts, raw materials, etc. Cranes are often operated manually, but automation of cranes has been proposed to improve production efficiency.

[0003] For example, Patent Document 1 proposes a crane system that automatically transports luggage. Specifically, the crane system disclosed in Patent Document 1 is composed of a sensor network terminal installed on the luggage, a base station terminal installed on the crane, and a control device.

[0004] The control device first receives the position information of the origin and destination of the load from the operator, and then inputs the position information to the crane, instructing the crane to transport the load.

[0005] Then, when the crane notifies the end of transportation, the control device identifies the position of each base station (crane) based on the position information of each of the multiple base station terminals. Furthermore, the control device identifies the position of the sensor network terminal, i.e., the position of the luggage, based on the reception time of each base station terminal of the wireless packet transmitted by the sensor network terminal and the position information of each base station terminal. The control device then determines whether the identified position of the luggage matches the position indicated by the input destination position information, and if they match, determines that transportation has ended. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2005-001804 A Summary of the Invention [Problem to be solved by the invention]

[0007] In this way, the crane system disclosed in Patent Document 1 can transport cargo to a specified location without the need for an operator to operate the crane. However, the crane system disclosed in Patent Document 1 has a problem in that it cannot adjust the position of the crane depending on the situation at the destination.

[0008] For example, assume that the cargo is a product manufactured in a factory, and the product is loaded at a predetermined position on the bed of a truck. In this case, the position of the truck is likely to vary, and the bed of the truck differs depending on the type of vehicle. Therefore, in order to accurately load the product on the bed of the truck, it is necessary to adjust the position of the crane according to the position of the bed. In addition, since a jig for fixing the cargo during transportation is usually installed on the bed of the truck, it is also necessary to adjust the position of the crane according to the position of the jig. Furthermore, the position of the jig may differ depending on the truck. For this reason, it is difficult for the crane system disclosed in the above Patent Document 1 to load cargo on a transport vehicle such as a truck.

[0009] An example of an object of the present invention is to provide a loading automation support device, a loading automation support method, and a program that can support the automation of loading by a transportation system. [Means for solving the problem]

[0010] In order to achieve the above object, a loading automation support device according to one aspect of the present invention comprises: A side data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving object; a baggage detection unit that detects a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using 3D point cloud data at a set height or higher among the 3D point cloud data acquired by the side data acquisition unit; and a jig detection unit that uses 3D point cloud data on the front side of the moving body of the detected surface of the baggage among the 3D point cloud data acquired by the side data acquisition unit to identify point cloud data on the top surface of a jig placed on the loading platform, and further detects the surface of the jig on the rear side of the moving body using the identified point cloud data; A distance calculation unit that calculates a distance between a detected surface of the luggage and a detected surface of the jig; The present invention is characterized in that it is equipped with:

[0011] In order to achieve the above object, a loading automation support method according to one aspect of the present invention includes: A side data acquisition step of acquiring 3D point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a baggage detection step of detecting a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using 3D point cloud data at a set height or higher among the 3D point cloud data acquired in the side data acquisition step; a jig detection step of identifying point cloud data on a top surface of a jig placed on the platform using 3D point cloud data on the front side of the moving body of the detected surface of the luggage among the 3D point cloud data acquired in the side data acquisition step, and further detecting the surface of the jig on the rear side of the moving body using the identified point cloud data; a distance calculation step of calculating a distance between the detected surface of the baggage and the detected surface of the jig; The present invention is characterized in that it has the following features:

[0012] Furthermore, in order to achieve the above object, a program according to one aspect of the present invention comprises: On the computer, A side data acquisition step of acquiring 3D point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a baggage detection step of detecting a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using 3D point cloud data at a set height or higher among the 3D point cloud data acquired in the side data acquisition step; a jig detection step of identifying point cloud data on a top surface of a jig placed on the platform using 3D point cloud data on the front side of the moving body of the detected surface of the luggage among the 3D point cloud data acquired in the side data acquisition step, and further detecting the surface of the jig on the rear side of the moving body using the identified point cloud data; a distance calculation step of calculating a distance between the detected surface of the baggage and the detected surface of the jig; The present invention is characterized in that: Effect of the Invention

[0013] As described above, according to the present invention, it is possible to assist in automating loading by a transport system. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a configuration diagram showing a schematic configuration of a loading automation support device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 shows an example of luggage and moving objects to be loaded in the first embodiment, where FIG. 2(a) is a side view and FIG. 2(b) is a top view. [Diagram 3] FIG. 3 is a diagram showing an example of a configuration of the loading automation support device according to the first embodiment and a transport system to be supported. [Figure 4] FIG. 4 is a diagram showing an example of three-dimensional point cloud data acquired from the side of a truck in the first embodiment. [Diagram 5] Figure 5 is a diagram for explaining the processing in the luggage detection unit in embodiment 1, where Figure 5(a) shows 3D point cloud data near the coil material from the side of the truck, Figure 5(b) shows the point cloud near the coil material projected onto the XY plane, and Figure 5(c) shows the surface of the detected coil material. [Figure 6]Figure 6 is a diagram for explaining the processing in the jig detection unit in embodiment 1, where Figure 6(a) shows the three-dimensional point cloud data extracted for jig detection from the top side of the track, and Figure 6(b) shows the three-dimensional point cloud data extracted for jig detection from the side side of the track. [Figure 7] Figure 7 is a diagram for explaining the processing in the distance calculation unit in embodiment 1, where Figure 7(a) shows the three-dimensional point cloud data representing the jig surface from the top side of the track, and Figure 7(b) shows the detected surface of the coil material and the jig surface. [Figure 8] FIG. 8 is a flow diagram showing the operation of the loading automation support device in the first embodiment. [Figure 9] FIG. 9 shows an example of luggage and moving objects to be loaded in the second embodiment, where FIG. 9(a) is a side view and FIG. 9(b) is a top view. [Figure 10] FIG. 10 is a diagram showing an example of a configuration of a loading automation support device according to the second embodiment and a transport system to be supported. [Figure 11] Figure 11 is a diagram showing an example of 3D point cloud data acquired from the rear in embodiment 2, where Figure 11(a) shows the entire 3D point cloud data acquired from the rear depth sensor from diagonally behind the depth sensor, and Figure 11(b) shows the 3D point cloud data corresponding to the truck from the top side of the truck. [Figure 12] Figure 12 is a diagram for explaining the processing in the loading platform detection unit in embodiment 2, where Figure 12(a) shows 3D point cloud data near the loading platform of a truck from the rear side of the truck, and Figure 12(b) shows 3D point cloud data of only the top surface of the loading platform extracted from the 3D point cloud data shown in Figure 12(a) from the top side of the truck. [Figure 13] FIG. 13 is a diagram for explaining the process of estimating the outer position of the skid on the bed in the second embodiment, and shows the three-dimensional point cloud data shown in FIG. 13 from the rear side of the truck. [Figure 14] FIG. 14 is a diagram for explaining the estimation process of a jig on a bed in the second embodiment, showing the bed of the truck from above the truck. [Figure 15] FIG. 15 is a diagram for explaining the process of estimating the inner position of the skid on the bed in the second embodiment, showing the bed of the truck from above the truck. [Figure 16] FIG. 16 is a diagram for explaining the process of estimating the height of the skid on the bed in the second embodiment, showing the bed of the truck from above the truck. [Figure 17] FIG. 17 is a diagram for explaining the process of searching for an obstruction on the bed in the second embodiment, showing the bed of the truck from above the truck. [Figure 18] FIG. 18 is a flow diagram showing the operation of the loading automation support device in the second embodiment. [Figure 19] FIG. 19 is a block diagram showing an example of a computer that realizes the loading automation support device according to the first and second embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] (Embodiment 1) Hereinafter, a loading automation support device, a loading automation support method, and a program according to the first embodiment will be described with reference to FIGS. 1 to 8. FIG.

[0016] [Device configuration] First, a schematic configuration of the loading automation support device in the first embodiment will be described with reference to Fig. 1. Fig. 1 is a configuration diagram showing a schematic configuration of the loading automation support device in the first embodiment.

[0017] The loading automation support device 10 in the first embodiment shown in Fig. 1 is a device for supporting loading by a luggage transport system. As shown in Fig. 1, the loading automation support device 10 includes a side data acquisition unit 11, a luggage detection unit 12, a tool detection unit 13, and a distance calculation unit 14.

[0018] The lateral data acquisition unit 11 acquires three-dimensional point cloud data generated from output data of a depth sensor. This depth sensor is disposed on the side of the moving object, and is hereinafter referred to as a "lateral depth sensor."

[0019] The luggage detection unit 12 detects the front surface of the luggage placed on the platform of the moving body using the 3D point cloud data that is above a set height among the 3D point cloud data acquired by the lateral data acquisition unit 11.

[0020] The jig detection unit 13 uses the 3D point cloud data on the front side of the moving body of the detected luggage surface from the 3D point cloud data acquired by the lateral data acquisition unit 11 to identify point cloud data on the top surface of the jig placed on the loading platform, and further detects the rear side of the moving body of the jig using the identified point cloud data.

[0021] The distance calculation unit 14 calculates the distance between the detected surface of the luggage and the detected surface of the jig. The loading automation support device 10 then transmits the calculated distance to the transportation system to be supported. The transportation system uses the transmitted distance to execute various controls required when loading luggage onto the platform of the mobile body.

[0022] As described above, the loading automation support device 10 can calculate the distance between the luggage on the platform of the moving body and the jig. The obtained distance is calculated using the 3D point cloud data obtained from the depth sensor, and is an accurate value. Therefore, the loading automation support device 10 can support the automation of loading in the transportation system.

[0023] Next, the luggage and moving objects to be loaded in the first embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of luggage and moving objects to be loaded in the first embodiment, Fig. 2(a) is a side view, and Fig. 2(b) is a top view.

[0024] 2(a) and (b), in the first embodiment, the cargo 20 is a cylindrical object, specifically, a coil material obtained by rolling a steel plate into a roll. Hereinafter, this will also be referred to as "coil material 20." A cubic or rectangular parallelepiped object is used as a jig 21 for placing the coil material 20.

[0025] In the first embodiment, the moving body 40 is a truck. Hereinafter, it will also be referred to as "truck 40". The truck 40 has a loading platform 41. In the first embodiment, the front side of the moving body will also be referred to as the truck front side, and the rear side of the moving body will also be referred to as the truck rear side.

[0026] The coil material 20 and the jig 21 are placed on the bed 41 of the truck 40. At this time, the coil material 20 is placed so that one end face constituting a cylinder faces the front side of the truck, and the other end face faces the rear side of the truck. The jig 21 is placed so that one face faces the front side of the truck, and the opposite face faces the rear side of the truck. In the example of Figures 2(a) and (b), a skid 22 for fixing the coil material is also placed on the bed 41. The skid is composed of two rail-shaped members.

[0027] Furthermore, as shown in Figures 2(a) and (b), in embodiment 1, the lateral depth sensor 31 is positioned on the side of the truck 40 so as to be able to sense the coil material 20 and the jig 21 arranged on the loading platform 41.

[0028] The lateral depth sensor 31 is a sensor capable of outputting a signal indicating the depth (distance) to an object. Examples of the lateral depth sensor 31 include a Time Of Flight (TOF) camera and a Light Detection and Ranging (LiDAR). In the example of FIGS. 2(a) and (b), the lateral depth sensor 31 is a three-dimensional LiDAR, and outputs three-dimensional point cloud data of a subject. As shown in FIG. 3 described later, the lateral depth sensor 31 is connected to the loading automation support device 10, and sends the three-dimensional point cloud data to the loading automation support device 10.

[0029] Next, the configuration of the loading automation support device 10 in the first embodiment will be specifically described with reference to Fig. 3. Fig. 3 is a diagram showing the configuration of the loading automation support device in the first embodiment and an example of a transport system to be supported. As shown in Fig. 3, the transport system 50 includes a crane 51 installed in a factory and a control device 52.

[0030] In the example of Fig. 3, the crane 51 is an overhead crane that moves along a runway installed near the ceiling of a factory. The type of crane 51 is not particularly limited in the first embodiment. The control device 52 controls the movement of the crane 51 itself, the movement of the hoist, and the up and down movement of the hook. For example, when the position of a load and the position of the destination of the load are specified, the control device 52 controls the crane 51 to transport the load from the specified position to the destination position.

[0031] 3, the loading automation support device 10 is connected to the control device 52 so as to be capable of data communication, and transmits information indicating the calculated distance between the surface of the coil material 20 and the surface of the jig 21. The control device 52 uses the transmitted information to control the crane 51.

[0032] Next, the functions of the loading automation support device 10 in the first embodiment will be specifically described with reference to Figs.

[0033] Fig. 4 is a diagram showing an example of three-dimensional point cloud data acquired from the side in the first embodiment, as viewed from the side of the truck. In the first embodiment, the side data acquisition unit 11 acquires three-dimensional point cloud data from a side depth sensor 31 arranged on the side of the truck 40. An example of the acquired three-dimensional point cloud data is as shown in the upper diagram of Fig. 4. In the first embodiment, in the three-dimensional point cloud data, the front-to-rear direction of the truck 40 (see Fig. 2) is defined as the "lateral direction: Y direction", the width direction of the truck 40 is defined as the "depth direction: X direction", and the height direction of the truck 40 is defined as the "height direction: Z direction".

[0034] Figure 5 is a diagram for explaining the processing in the luggage detection unit in embodiment 1, where Figure 5(a) shows 3D point cloud data near the coil material from the side of the truck, Figure 5(b) shows the point cloud near the coil material projected onto the XY plane, and Figure 5(c) shows the surface of the detected coil material.

[0035] In the first embodiment, the baggage detection unit 12 first extracts 3D point cloud data in the near distance area from the 3D point cloud data acquired by the side data acquisition unit 11. Specifically, as shown in Fig. 4, the baggage detection unit 12 extracts 3D point cloud data composed only of points whose depth values ​​are equal to or less than a set value. This removes point clouds of objects located outside the bed of the truck 40. The area surrounded by a dashed frame in the upper diagram of Fig. 4 is the extracted area. The lower diagram of Fig. 4 shows only the extracted 3D point cloud data.

[0036] Next, the baggage detection unit 12 converts the extracted 3D point cloud data into voxels, although this is not shown in Fig. 4. The voxelization is performed by, for example, setting one side of each box to 1 cm, and removing boxes with 4 or less points as noise. Boxes that have not been removed as noise are considered to be valid boxes.

[0037] Furthermore, as shown in Fig. 5(a), the baggage detection unit 12 identifies 3D point cloud data (a portion surrounded by a dashed frame line) in the upper half of the effective box of the 3D point cloud data extracted in Fig. 4. Then, as shown in Fig. 5(b), the baggage detection unit 12 projects the identified 3D point cloud data onto the XY plane.

[0038] Next, the luggage detection unit 12 performs a Hough transform on the point cloud projected onto the XY plane shown in FIG. 5(b) to detect multiple candidates for line segments on the XY plane that may be the surface of the coil material 20 (see FIG. 2). Next, the luggage detection unit 12 determines a line segment that corresponds to the surface of the coil material 20 from among the multiple candidates in accordance with set conditions. Then, the luggage detection unit 12 detects a surface that includes the determined line segment and is perpendicular to the XY plane as the surface of the coil material 20. Examples of set conditions include a small angle with respect to an axis along the Y direction and a long straight portion. The surface detected at this time is the surface of the coil material on the front side of the moving body.

[0039] Figure 6 is a diagram for explaining the processing in the jig detection unit in embodiment 1, where Figure 6(a) shows the three-dimensional point cloud data extracted for jig detection from the top side of the track, and Figure 6(b) shows the three-dimensional point cloud data extracted for jig detection from the side side of the track.

[0040] In embodiment 1, the jig detection unit 13 first identifies the frontmost (depth sensor side) point on the surface of the coil material 20 detected by the luggage detection unit 12 (hereinafter referred to as the ``front end point of the coil surface''), and estimates that the position a radius (default value) of the coil material 20 behind the identified front end point of the coil surface is the center of the coil material 20 (see Figure 5 (c)).

[0041] 6(a) and (b), the jig detection unit 13 extracts 3D point cloud data within a predetermined range based on the estimated center of the coil material 20 from the 3D point cloud data (see the lower diagram in FIG. 4) extracted and voxelized by the baggage detection unit 12. The predetermined range is appropriately set according to the size of the jig 21.

[0042] Furthermore, the jig detection unit 13 searches the extracted three-dimensional point cloud data along the Y direction to detect the upper surface of the jig 21 (see FIG. 2). Specifically, as shown in FIG. 6(b), the jig detection unit 13 identifies, for example, a location where the difference between the height of the front side of the track and the height of the rear side of the track is α cm or more, that is, a location where there is a step. Next, for example, if an area where the height is within ±β cm continues for γ cm or more on the front side of the track from the identified location, the jig detection unit 13 detects this area as the upper surface of the jig 21. The values ​​of α, β, and γ are also appropriately set according to the size of the jig 21. Then, when the jig detection unit 13 determines the upper surface of the jig 21, it identifies a surface perpendicular to the XY plane at the location where there is a step, and detects the identified surface as the surface of the jig. The surface detected at this time is the surface of the jig 21 on the rear side of the moving body.

[0043] Figure 7 is a diagram for explaining the processing in the distance calculation unit in embodiment 1, where Figure 7(a) shows the three-dimensional point cloud data representing the jig surface from the top side of the track, and Figure 7(b) shows the detected surface of the coil material and the jig surface.

[0044] In the first embodiment, the distance calculation unit 14 projects three-dimensional point cloud data in a predetermined range centered on the surface of the jig onto the XY plane as shown in Fig. 7(a). Next, if there is an opening in the projected point cloud, the distance calculation unit 14 adds a point cloud to the location of the opening. Then, as shown in Fig. 7(b), the distance calculation unit 14 identifies two end points in the Y direction of the projected point cloud, and calculates the distance between each end point and the surface of the coil material detected by the baggage detection unit 12. After that, the distance calculation unit 14 determines the shorter distance of the distances for each end point as the distance between the surface of the coil material and the surface of the jig.

[0045] When the distance calculation by the distance calculation unit 14 is completed, as described above, the loading automation support device 10 transmits information indicating the calculated distance between the surface of the package and the surface of the jig to the control device 52. As a result, the control device 52 uses the transmitted information for controlling the crane 51.

[0046] [Device operation] Next, the operation of the loading automation support device 10 in the first embodiment will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing the operation of the loading automation support device 10 in the first embodiment. In the following description, Figs. 1 to 7 will be referred to as appropriate. Also, in the first embodiment, a loading automation support method is implemented by operating the loading automation support device 10. Therefore, the description of the loading automation support method in the first embodiment will be replaced with the following description of the operation of the loading automation support method.

[0047] As shown in FIG. 8, first, the side data acquisition unit 11 acquires three-dimensional point cloud data from the side depth sensor 31 arranged on the side of the truck 40 (step A1).

[0048] Next, the luggage detection unit 12 detects the surface of the luggage placed on the platform of the moving body facing forward, using point cloud data that is at or above a set height from the 3D point cloud data acquired in step A1 (step A2).

[0049] Specifically, in step A2, the luggage detection unit 12 extracts 3D point cloud data in the close distance area from the 3D point cloud data acquired by the lateral data acquisition unit 11, as shown in FIG. 4, and further converts the extracted 3D point cloud data into voxels.

[0050] Furthermore, as shown in Figures 5(a) and (b), the baggage detection unit 12 identifies the 3D point cloud data of the upper half of the effective box of the 3D point cloud data extracted and voxelized in step A2, and projects the identified 3D point cloud data onto the XY plane.

[0051] Then, the baggage detection unit 12 performs a Hough transform on the point cloud projected onto the XY plane to detect multiple candidates for line segments on the XY plane that may become the surface of the coil material 20, and determines the line segment that corresponds to the surface of the coil material from among the multiple candidates in accordance with the set conditions. After that, the baggage detection unit 12 detects a surface that includes the determined line segment and is perpendicular to the XY plane as the surface of the coil material 20.

[0052] Next, the jig detection unit 13 uses the 3D point cloud data on the front side of the truck of the surface of the detected coil material 20 from the 3D point cloud data acquired in step A1 to identify the point cloud data on the top surface of the jig 21 placed on the loading platform, and further uses the identified point cloud data to detect the surface of the jig on the rear side of the moving body (step A3).

[0053] Specifically, in step A3, the jig detection unit 13 estimates the center of the coil material 20, as shown in Figure 5(c), and extracts 3D point cloud data within a predetermined range based on the center of the coil material 20 from the 3D point cloud data extracted and voxelized in step A2, as shown in Figures 6(a) and (b).

[0054] Furthermore, the jig detection unit 13 searches the 3D point cloud data extracted in step A5 along the Y direction to detect the top surface of the jig 21, and based on the detected top surface, identifies a surface perpendicular to the XY plane at the location of the step, and detects the identified surface as the surface of the jig.

[0055] Next, distance calculation unit 14 calculates the distance between the surface of coil material 20 detected in step A2 and the surface of jig 21 detected in step A3 (step A4).

[0056] Specifically, in step A4, distance calculation unit 14 projects 3D point cloud data in a predetermined range centered on the surface of jig 21 onto the XY plane, as shown in Fig. 7(a), and identifies two end points in the Y direction of the projected point cloud. Then, distance calculation unit 14 calculates the distance between the surface of coil material 20 and the surface of jig 21 using the distance between each end point and the surface of coil material 20.

[0057] Next, the distance calculation unit 14 transmits information indicating the distance between the surface of the coil material 20 and the surface of the jig 21 calculated in step A4 to the control device 52 (step A5). As a result, the control device 52 uses the transmitted information for controlling the crane 51.

[0058] As described above, the loading automation support device 10 calculates the distance between the coil material 20 on the bed of the truck 40 and the jig 21, and transmits this to the control device 52. The distance is calculated using the three-dimensional point cloud data output by the LiDAR, which is the lateral depth sensor 31, and the calculated value is accurate. Therefore, the loading automation support device 10 can support the automation of the loading of the coil material 20 onto the truck 40 by the transport system 50.

[0059] [program] The program in the first embodiment may be any program that causes a computer to execute steps A1 to A5 shown in Fig. 8. By installing and executing this program in a computer, the loading automation support device 10 and the loading automation support method in the first embodiment can be realized. In this case, the processor of the computer functions as the side data acquisition unit 11, the baggage detection unit 12, the tool detection unit 13, and the distance calculation unit 14 to perform processing. Examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0060] The program in the first embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the lateral data acquisition unit 11, the baggage detection unit 12, the jig detection unit 13, and the distance calculation unit 14.

[0061] (Embodiment 2) Next, a loading automation support device, a loading automation support method, and a program according to the second embodiment will be described with reference to FIGS.

[0062] [Device configuration] First, the configuration of the loading automation support device in the second embodiment will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is a diagram showing an example of luggage and moving objects to be loaded in the second embodiment, Fig. 9(a) is a side view, and Fig. 9(b) is a top view. Fig. 10 is a diagram showing the configuration of the loading automation support device in the second embodiment and an example of a transportation system to be supported.

[0063] 9(a) and (b), in the second embodiment, the cargo 20 is a coil material, as in the first embodiment. A cubic or rectangular parallelepiped object is used as the jig 21. The jig 21 is arranged so that one surface faces the front side of the moving body and the opposite surface faces the rear side of the moving body. Furthermore, in the second embodiment, the moving body 40 is a truck, as in the first embodiment.

[0064] In addition, as shown in Figures 9(a) and (b), in embodiment 2 as well, a lateral depth sensor 31 is positioned on the side of the truck 40 so as to be able to sense the coil material 20 and the jig 21 placed on the loading platform 41.

[0065] However, in the second embodiment, unlike the first embodiment, a depth sensor 32 is also installed at the rear of the truck 40. As the depth sensor (hereinafter referred to as the "rear depth sensor") 32, a three-dimensional LiDAR is used, similar to the lateral depth sensor 31. The rear depth sensor 32 outputs three-dimensional point cloud data obtained by sensing from the rear of the truck. In addition, as shown in FIG. 10 described later, the rear depth sensor 32 is connected to the loading automation support device 60 in the second embodiment, and sends three-dimensional point cloud data from the rear of the truck to the loading automation support device 60.

[0066] As shown in Fig. 10, the loading automation support device 60 in the second embodiment is a device for supporting loading in the transport system 50, similar to the first embodiment. In the second embodiment, the transport system 50 includes a crane 51 installed in a factory, a control device 52, and a position adjustment device 53. The position adjustment device 53 is a device that adjusts the position of the side depth sensor 31 arranged on the side of the truck. Although not shown in Fig. 10, the position adjustment device 53 includes a movement mechanism such as a ball screw, a driving electric motor, etc.

[0067] The crane 51 is an overhead crane that moves along a runway installed near the ceiling of the factory, as in the first embodiment. The control device 52 is connected to the loading automation support device 60 so as to be able to communicate data with it, as in the first embodiment. The control device 52 uses information transmitted from the loading automation support device 60 to control the crane 51. In the second embodiment, the control device 52 controls the position adjustment device 53 based on information transmitted from the loading automation support device 60, as will be described later.

[0068] As shown in FIG. 10, the loading automation support device 60 includes a lateral data acquisition unit 11, a luggage detection unit 12, a jig detection unit 13, a distance calculation unit 14, a rear data acquisition unit 15, a moving object identification unit 16, and a loading platform detection unit 17.

[0069] Among these, the side data acquisition unit 11, the baggage detection unit 12, the jig detection unit 13, and the distance calculation unit 14 are configured in the same manner as in the first embodiment, and further have the same functions. Therefore, in the second embodiment, the description of the side data acquisition unit 11, the baggage detection unit 12, the jig detection unit 13, and the distance calculation unit 14 will be omitted.

[0070] The rear data acquisition unit 15 acquires three-dimensional point cloud data from a rear depth sensor 32 disposed behind a moving body, that is, a truck 40. The moving body identification unit 16 identifies point cloud data corresponding to the truck 40 from the three-dimensional point cloud data acquired by the rear data acquisition unit 15.

[0071] The loading platform detection unit 17 uses the point cloud data identified by the moving body identification unit 16 to identify the height of the loading platform of the truck 40, and detects an area on the loading platform 41 in which the coil material 20 can be loaded, using point cloud data that is above the identified loading platform height.

[0072] Here, the function of the loading automation support device 60 in the second embodiment will be specifically described with reference to Figs.

[0073] Figure 11 is a diagram showing an example of 3D point cloud data acquired from the rear in embodiment 2, where Figure 11(a) shows the entire 3D point cloud data acquired from the rear depth sensor from diagonally behind the depth sensor, and Figure 11(b) shows the 3D point cloud data corresponding to the truck from the top side of the truck.

[0074] In the second embodiment, the rear data acquisition unit 15 acquires three-dimensional point cloud data from a rear depth sensor 32 arranged behind the moving body 40. An example of the acquired three-dimensional point cloud data is as shown in Fig. 11(a). In the second embodiment, in the three-dimensional point cloud data, the front-to-rear direction of the truck 40 (see Fig. 9) is defined as the "depth direction: Y direction", the width direction of the truck 40 is defined as the "lateral direction: X direction", and the height direction of the truck 40 is defined as the "height direction: Z direction".

[0075] The moving object identification unit 16 identifies the three-dimensional point cloud data corresponding to the truck 40 shown in Fig. 11(b) from the three-dimensional point cloud data shown in Fig. 11(a). Specifically, the moving object identification unit 16 extracts the three-dimensional point cloud data existing in a preset area. The area to be extracted is appropriately set according to the size of the truck 40, the position of the parking area, etc.

[0076] Figure 12 is a diagram for explaining the processing in the loading platform detection unit in embodiment 2, where Figure 12(a) shows 3D point cloud data near the loading platform of a truck from the rear side of the truck, and Figure 12(b) shows 3D point cloud data of only the top surface of the loading platform extracted from the 3D point cloud data shown in Figure 12(a) from the top side of the truck.

[0077] First, the loading platform detection unit 17 generates a projection histogram in the height direction (Z direction) of the truck 40 (see FIG. 9) using the 3D point cloud data identified by the moving object identification unit 16. Next, the loading platform detection unit 17 calculates the loading platform height, the bottom end of the object on the loading platform, and the top end of the object on the loading platform based on the projection histogram.

[0078] Specifically, for example, the platform detection unit 17 identifies the location with the maximum number of votes among all votes in the projection histogram, and sets the height of the identified location as the platform height. The platform detection unit 17 also adds a predetermined value to the set platform height, sets the obtained value as the lower end of the object on the platform, and adds another predetermined value to the set lower end of the object on the platform, and sets the obtained value as the upper end of the object on the platform. The predetermined value used in this case is appropriately set according to the height of the jig 21 and the skid 22, and further the thickness of the members arranged on the platform 41, etc.

[0079] Next, the loading platform detection unit 17 extracts 3D point cloud data between the lower end of the object on the loading platform and the upper end of the object on the loading platform. The extracted 3D point cloud data is as shown in Fig. 12(b). Similarly to the baggage detection unit 12 described in the first embodiment, the loading platform detection unit 17 can convert the extracted 3D point cloud data into voxels to remove noise.

[0080] FIG. 13 is a diagram for explaining the process of estimating the outer position of the skid on the bed in the second embodiment, and shows the three-dimensional point cloud data shown in FIG. 13 from the rear side of the truck.

[0081] The loading platform detection unit 17 generates a projection histogram in the width direction (X direction) of the truck using 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform. Next, the loading platform detection unit 17 divides the projection histogram at the center and calculates the average height in each of the left and right parts in the figure.

[0082] The loading platform detection unit 17 also searches this projection histogram from the left side in the figure, and sets the first point where the height exceeds the average height to the outer position of the skid 22 on the left side in the figure. The loading platform detection unit 17 also searches this projection histogram from the right side in the figure, and sets the first point where the height exceeds the average height to the outer position of the skid 22 on the right side in the figure.

[0083] Fig. 14 is a diagram for explaining the estimation process of the jig on the bed in the second embodiment, and shows the bed of the truck from above the truck. In Fig. 14, a jig 21 and a skid 22 are arranged on bed 41 of truck 40.

[0084] As shown in Fig. 14, the loading platform detection unit 17 uses the 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform to set, for example, regions n1 to n5 in the portion surrounded by the right and left skids 22, and generates a projection histogram in the front-rear direction (Y direction) of the truck for each region. Then, the loading platform detection unit 17 sets, for example, a region in the projection histogram where the value is larger on the front side of the truck than on the rear side as a candidate region for the jig 21. Note that, although the number of regions is set to five in the example of Fig. 14, the number of regions is not particularly limited.

[0085] Next, the loading platform detection unit 17 generates a projection histogram in the height direction (Z direction) of the truck in the candidate area, and, for example, sets the part of this projection histogram that continues from the maximum value as the upper surface of the jig 21, and sets the height of this part as the height of the jig 21.

[0086] Furthermore, the loading platform detection unit 17 generates a projection histogram in the width direction (X direction) of the truck in the candidate area, and sets the length of the continuous portion from the maximum value in this projection histogram to the width of the jig 21, for example.

[0087] In addition, the loading platform detection unit 17 generates a projection histogram in the fore-and-aft direction (Y direction) of the truck in an area obtained by expanding the candidate area by a predetermined value in the fore-and-aft direction of the truck, and sets the position of the maximum value in this projection histogram as the position of the end face of the jig 21.

[0088] Fig. 15 is a diagram for explaining the process of estimating the inner position of the skid on the bed in the second embodiment, and shows the bed of the truck from above the truck. In Fig. 15 as well, a jig 21 and a skid 22 are arranged on bed 41 of truck 40.

[0089] 15, the loading platform detection unit 17 sets the area A so that the area A is wider on the left and right than the outside of the skid, with the position of the end face of the set jig 21 as the innermost edge. Furthermore, the loading platform detection unit 17 obtains a projection histogram in the width direction (X direction) of the truck using the 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform.

[0090] Next, the loading platform detection unit 17 searches this projection histogram from the center to the left, and sets the first point where the height exceeds the average height to the inside position of the skid 22 on the left side of the figure. The loading platform detection unit 17 also searches this average projection histogram from the center to the right, and sets the first point where the height exceeds the average height to the inside position of the skid 22 on the right side of the figure.

[0091] Fig. 16 is a diagram for explaining the process of estimating the height of the skid on the bed in the second embodiment, and shows the bed of the truck from above the truck. In Fig. 16 as well, a jig 21 and a skid 22 are placed on bed 41 of truck 40.

[0092] The loading platform detection unit 17 sets a plurality of regions on the skid 22, as shown in Fig. 16. In Fig. 16, a frame-like dashed line indicates the set regions. Next, the loading platform detection unit 17 identifies the maximum height for each region using 3D point cloud data between the lower end of the object on the loading platform and the upper end of the object on the loading platform, calculates the average value of the maximum height for each of the left and right skids 22, and sets the calculated average value as the height of each skid 22.

[0093] Fig. 17 is a diagram for explaining the process of searching for an obstruction on the bed in the second embodiment, showing the bed of the truck from above the truck. In Fig. 17 as well, a jig 21 and a skid 22 are arranged on bed 41 of truck 40.

[0094] As shown in Fig. 17, the loading platform detection unit 17 sets five areas B1 to B5 on the loading platform 41, and searches for objects higher than the reference height of each area using 3D point cloud data between the lower end of the object on the loading platform and the upper end of the object on the loading platform. In areas B1 and B2, the reference height is the height of the right skid 22. In area B3, the reference height is the height of the loading platform 41. In areas B4 and B5, the reference height is the height of the left skid 22. The lengths of areas B1 to B5 are set appropriately depending on the size of the coil material.

[0095] Then, the loading platform detector 17 transmits information on the detected area in which the coil material can be loaded to the control device 52. Specifically, the information includes the outer center positions of the left and right skids 22, the position of the end face (the face on the rear side of the truck) of the jig 21, the height of the loading platform 41, and the presence or absence of an obstruction.

[0096] In the transport system 50, the control device 52 controls the position adjustment device 53 based on the transmitted information to adjust the position of the side depth sensor 31 on the side of the truck 40. As a result, the side depth sensor 31 is disposed on the side of the truck 40 so as to be able to sense the coil material 20 and the jig 21 disposed on the loading platform 41. Then, when the coil material 20 is temporarily placed on the loading platform 41 and the end of the temporary placement is input, the control device 52 notifies the loading automation support device 60 that the temporary placement is completed. Thereafter, the loading automation support device 60 also executes the process shown in the first embodiment. As a result, also in the second embodiment, the loading automation support device 60 transmits information indicating the calculated distance between the surface of the baggage 20 and the surface of the jig 21 to the control device 52.

[0097] [Device operation] Next, the operation of the loading automation support device 60 in the second embodiment will be described with reference to Fig. 18. Fig. 18 is a flow diagram showing the operation of the loading automation support device 60 in the second embodiment. In the following description, Figs. 9 to 17 will be referred to as appropriate. Also, in the twenty-first embodiment, a loading automation support method is implemented by operating the loading automation support device 60. Therefore, the description of the loading automation support method in the second embodiment will be replaced with the following description of the operation of the loading automation support method.

[0098] As shown in FIG. 18, first, the rear data acquisition unit 15 acquires three-dimensional point cloud data from the rear depth sensor 32 disposed at the rear of the truck 40 (step B1).

[0099] Next, the moving object identifying unit 16 identifies point cloud data corresponding to the truck 40 from the three-dimensional point cloud data acquired in step B1 (step B2). Specifically, in step B2, the moving object identifying unit 16 extracts three-dimensional point cloud data existing in a preset area.

[0100] Next, the bed detection unit 17 uses the point cloud data identified in step B2 to identify the height of the bed 41 of the truck 40, and detects an area on the bed 41 in which the coil material 20 can be loaded, using point cloud data that is above the identified bed height (step B3).

[0101] Specifically, in step B3, the loading platform detection unit 17 first calculates the loading platform height of the truck 40, the bottom end of the object on the loading platform, and the top end of the object on the loading platform, as shown in Fig. 12, using the 3D point cloud data identified in step B2. Furthermore, the loading platform detection unit 17 extracts 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform.

[0102] In step B3, next, the loading platform detection unit 17 estimates the outer positions of the left and right skids on the loading platform 41 of the truck 40 using the 3D point cloud data between the lower end of the object on the loading platform and the upper end of the object on the loading platform as shown in Fig. 13. Furthermore, the loading platform detection unit 17 specifies the height, width, and end face of the jig 21 using the 3D point cloud data between the upper end of the object on the loading platform as shown in Fig. 14.

[0103] In step B3, the loading platform detection unit 17 then estimates the inner positions of the left and right skids on the loading platform 41 of the truck 40 using the three-dimensional point cloud data between the bottom end of the object on the platform and the top end of the object on the platform, as shown in Fig. 15. The loading platform detection unit 17 also identifies the height of the skids using the three-dimensional point cloud data between the bottom end of the object on the platform and the top end of the object on the platform, as shown in Fig. 16. Furthermore, the loading platform detection unit 17 searches for an interfering object on the loading platform using the three-dimensional point cloud data between the bottom end of the object on the platform and the top end of the object on the platform, as shown in Fig. 17.

[0104] Next, after step B3 is completed, the loading platform detector 17 transmits information on the detected area in which the coil material can be loaded to the controller 52 (step B4). The information transmitted in step B4 includes the outer center positions of the left and right skids 22, the position of the end face (the face on the rear side of the truck) of the jig 21, the height of the loading platform 41, and the presence or absence of an obstruction.

[0105] When step B4 is executed, in the transportation system 50, the control device 52 controls the position adjustment device 53 based on the transmitted information to adjust the position of the side depth sensor 31 on the side of the truck 40. In addition, when the coil material 20 is temporarily placed on the loading platform 41, the control device 52 notifies the loading automation support device 60 of the completion of the temporary placement.

[0106] After executing step B4, the loading platform detection unit 17 judges whether or not the completion of the temporary placement has been notified by the control device 52 (step B5). If the result of the judgment in step B5 is that the completion of the temporary placement has not yet been notified, the loading platform detection unit 17 goes into a standby state and executes step B5 again at set time intervals.

[0107] On the other hand, if the result of the determination in step B5 indicates that the temporary placement is complete, the side data acquisition unit 11 acquires three-dimensional point cloud data from the side depth sensor 31 arranged on the side of the truck 40 (step B6). Step B1 is the same as step A1 shown in FIG.

[0108] Next, the baggage detection unit 12 detects the surface of the baggage placed on the platform of the moving body on the front side of the moving body by using the point cloud data at or above the set height among the 3D point cloud data acquired in step B6 (step B7). Step B7 is the same as step A2 shown in FIG.

[0109] Next, the jig detection unit 13 uses the 3D point cloud data on the front side of the truck of the surface of the detected coil material 20 among the 3D point cloud data acquired in step B6 to identify the point cloud data on the top surface of the jig placed on the loading platform, and further detects the surface of the jig 21 on the rear side of the moving body using the identified point cloud data (step B8). Step B8 is a step similar to step A3 shown in FIG. 8.

[0110] Next, the distance calculation unit 14 calculates the distance between the surface of the coil material 20 detected in step B7 and the surface of the jig 21 detected in step B8 (step B9). Step B9 is the same as step A4 shown in FIG.

[0111] Next, the distance calculation unit 14 transmits information indicating the distance between the surface of the coil material 20 and the surface of the jig 21 calculated in step B9 to the control device 52 (step B10). As a result, the control device 52 uses the transmitted information for controlling the crane 51.

[0112] As described above, according to the loading automation support device 60, the position of the lateral depth sensor 31 arranged on the side is automatically adjusted, and the distance between the coil material 20 on the bed of the truck 40 and the jig 21 is calculated. According to the second embodiment, it is possible to further support the automation of the loading of the coil material 20 onto the truck 40 by the transport system 50.

[0113] [program] The program in the second embodiment may be a program that causes a computer to execute steps B1 to B10 shown in Fig. 18. By installing and executing this program in a computer, the loading automation support device 60 and the loading automation support method in the second embodiment can be realized. In this case, the processor of the computer functions as the side data acquisition unit 11, the baggage detection unit 12, the jig detection unit 13, the distance calculation unit 14, the rear data acquisition unit 15, the moving object identification unit 16, and the loading platform detection unit 17, and performs processing. Examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0114] The program in the second embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of the side data acquisition unit 11, the baggage detection unit 12, the jig detection unit 13, the distance calculation unit 14, the rear data acquisition unit 15, the moving object identification unit 16, and the loading platform detection unit 17.

[0115] (physical configuration) Here, a computer that realizes the loading automation support device by executing the programs in the first and second embodiments will be described with reference to Fig. 19. Fig. 19 is a block diagram showing an example of a computer that realizes the loading automation support device in the first and second embodiments of the present invention.

[0116] 19, a computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0117] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this embodiment, the GPU or FPGA can execute the programs in the embodiments.

[0118] The CPU 111 loads the program in the first or second embodiment, which is composed of a group of codes and is stored in the storage device 113, into the main memory 112, and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0119] Moreover, the program in the first or second embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the present embodiment may be distributed on the Internet connected via the communication interface 117.

[0120] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0121] Data reader / writer 116 mediates data transmission between CPU 111 and recording medium 120, reads programs from recording medium 120, and writes processing results in computer 110 to recording medium 120. Communication interface 117 mediates data transmission between CPU 111 and other computers.

[0122] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as a flexible disk, or optical recording media such as a CD-ROM (Compact Disk Read Only Memory).

[0123] The loading automation support device in the first and second embodiments can be realized by using hardware corresponding to each part, instead of a computer with a program installed. Furthermore, the loading automation support device may be realized in part by a program and the remaining part by hardware.

[0124] A part or all of the above-described embodiment can be expressed by (Additional Notes 1) to (Additional Notes 12) described below, but is not limited to the following descriptions.

[0125] (Appendix 1) A side data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving object; a baggage detection unit that detects a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using 3D point cloud data at a set height or higher among the 3D point cloud data acquired by the side data acquisition unit; and a jig detection unit that uses 3D point cloud data on the front side of the moving body of the detected surface of the baggage among the 3D point cloud data acquired by the side data acquisition unit to identify point cloud data on the top surface of a jig placed on the loading platform, and further detects the surface of the jig on the rear side of the moving body using the identified point cloud data; A distance calculation unit that calculates a distance between a detected surface of the luggage and a detected surface of the jig; Equipped with A loading automation support device characterized by the above.

[0126] (Appendix 2) 2. A loading automation support device according to claim 1, A rear data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving body; a moving object identification unit that identifies three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition unit; a platform detection unit that uses the point cloud data identified by the moving body identification unit to identify a platform height of the moving body, and detects an area on the platform where the luggage can be loaded, using point cloud data that is at or above the identified platform height; The device further comprises: A loading automation support device characterized by the above.

[0127] (Appendix 3) 3. A loading automation support device according to claim 2, The position of the depth sensor arranged on the side of the moving body is adjusted based on the area in which the luggage can be carried detected by the loading platform detection unit, and then the side data acquisition unit acquires the three-dimensional point cloud data. A loading automation support device characterized by the above.

[0128] (Appendix 4) A loading automation support device according to any one of appendixes 1 to 3, The baggage is a cylindrical object, and is placed on the loading platform such that one end surface of the cylinder faces the front side of the moving body and the other end surface faces the rear side of the moving body; The jig is an object having a cube or a rectangular parallelepiped shape, and is placed on the platform so that one surface of the cube or the rectangular parallelepiped faces the front side of the moving body and the opposite surface faces the rear side of the moving body. A loading automation support device characterized by the above.

[0129] (Appendix 5) A side data acquisition step of acquiring 3D point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a baggage detection step of detecting a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using 3D point cloud data at a set height or higher among the 3D point cloud data acquired in the side data acquisition step; a jig detection step of identifying point cloud data on a top surface of a jig placed on the platform using 3D point cloud data on the front side of the moving body of the detected surface of the luggage among the 3D point cloud data acquired in the side data acquisition step, and further detecting the surface of the jig on the rear side of the moving body using the identified point cloud data; a distance calculation step of calculating a distance between the detected surface of the baggage and the detected surface of the jig; having A method for supporting automated loading, comprising:

[0130] (Appendix 6) A method for assisting in automating loading according to claim 5, comprising: A rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving body; a moving object identification step of identifying 3D point cloud data corresponding to the moving object from the 3D point cloud data acquired by the rear data acquisition step; a platform detection step of identifying a platform height of the moving body using the point cloud data identified by the moving body identification step, and detecting an area on the platform in which the luggage can be loaded using point cloud data that is at or above the identified platform height; Further comprising A method for supporting automated loading, comprising:

[0131] (Appendix 7) A method for assisting in automation of loading according to claim 6, comprising: a position of the depth sensor disposed on a side of the moving body is adjusted based on the area in which the luggage can be carried detected in the loading platform detection step, and then the 3D point cloud data is acquired in the side data acquisition step; A method for supporting automated loading, comprising:

[0132] (Appendix 8) A method for supporting automated loading according to any one of appendices 5 to 7, The baggage is a cylindrical object, and is placed on the loading platform such that one end surface of the cylinder faces the front side of the moving body and the other end surface faces the rear side of the moving body; The jig is an object having a cube or a rectangular parallelepiped shape, and is placed on the platform so that one surface of the cube or the rectangular parallelepiped faces the front side of the moving body and the opposite surface faces the rear side of the moving body. A method for supporting automated loading, comprising:

[0133] (Appendix 9) On the computer, A side data acquisition step of acquiring 3D point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a baggage detection step of detecting a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using 3D point cloud data at a set height or higher among the 3D point cloud data acquired in the side data acquisition step; a jig detection step of identifying point cloud data on a top surface of a jig placed on the platform using 3D point cloud data on the front side of the moving body of the detected surface of the luggage among the 3D point cloud data acquired in the side data acquisition step, and further detecting the surface of the jig on the rear side of the moving body using the identified point cloud data; a distance calculation step of calculating a distance between the detected surface of the baggage and the detected surface of the jig; A program to execute.

[0134] (Appendix 10) 10. The program of claim 9, The computer includes: A rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving body; a moving object identification step of identifying 3D point cloud data corresponding to the moving object from the 3D point cloud data acquired by the rear data acquisition step; a platform detection step of identifying a platform height of the moving body using the point cloud data identified by the moving body identification step, and detecting an area on the platform in which the luggage can be loaded using point cloud data that is at or above the identified platform height; Further execute A program characterized by:

[0135] (Appendix 11) 11. The program of claim 10, a position of the depth sensor disposed on a side of the moving body is adjusted based on the area in which the luggage can be carried detected in the loading platform detection step, and then the 3D point cloud data is acquired in the side data acquisition step; A program characterized by:

[0136] (Appendix 12) The program according to any one of appendices 9 to 11, The baggage is a cylindrical object, and is placed on the loading platform such that one end surface of the cylinder faces the front side of the moving body and the other end surface faces the rear side of the moving body; The jig is an object having a cube or a rectangular parallelepiped shape, and is placed on the platform so that one surface of the cube or the rectangular parallelepiped faces the front side of the moving body and the opposite surface faces the rear side of the moving body. A program characterized by: [Industrial Applicability]

[0137] INDUSTRIAL APPLICABILITY As described above, the present invention can assist in the automation of loading by a transport system. The present invention is useful in a system that promotes the automation of luggage transport. [Explanation of symbols]

[0138] 10. Loading automation support device 11. Lateral data acquisition section 12 Baggage detection unit 13 Jig detection section 14 Distance calculation section 15 Rear data acquisition section 16 Mobile Identification Unit 17 Cargo platform detector 20 Luggage (coil material) 21 Jig 22 Skid 31 Depth Sensor 32 Depth Sensor 40 Mobile (truck) 41 Cargo Bed 50 Transportation System 51 Crane 52 Control device 53 Position adjustment device 60 Loading automation support device (embodiment 2) 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Reader / Writer 117 Communication Interface 118 Input Devices 119 Display device 120 Recording media 121 Bus

Claims

1. A side data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving object; a baggage detection unit that detects a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using three-dimensional point cloud data that is at a set height or higher among the three-dimensional point cloud data acquired by the side data acquisition unit; and a jig detection unit that uses the three-dimensional point cloud data acquired by the side data acquisition unit that is on the front side of the moving body of the detected surface of the baggage to identify point cloud data on the top surface of a jig placed on the platform, and further detects the surface of the jig on the rear side of the moving body using the identified point cloud data; A distance calculation unit that calculates a distance between a detected surface of the luggage and a detected surface of the jig; Equipped with A loading automation support device characterized by the above.

2. 2. The loading automation support device according to claim 1, a rear data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving body; a moving object identification unit that identifies three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition unit; a platform detection unit that uses the point cloud data identified by the moving body identification unit to identify a platform height of the moving body, and detects an area on the platform where the luggage can be loaded, using point cloud data that is at or above the identified platform height; The device further comprises: A loading automation support device characterized by the above.

3. 3. The loading automation support device according to claim 2, a position of the depth sensor disposed on a side of the moving body is adjusted based on the area in which the luggage can be carried detected by the loading platform detection unit, and then the side data acquisition unit acquires the three-dimensional point cloud data; A loading automation support device characterized by the above.

4. The loading automation support device according to any one of claims 1 to 3, The baggage is a cylindrical object, and is placed on the loading platform such that one end surface of the cylinder faces the front side of the moving body and the other end surface faces the rear side of the moving body; The jig is an object having a cube or a rectangular parallelepiped shape, and is placed on the platform so that one surface of the cube or the rectangular parallelepiped faces the front side of the moving body and the opposite surface faces the rear side of the moving body. A loading automation support device characterized by the above.

5. A side data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a baggage detection step of detecting a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using the three-dimensional point cloud data at a height equal to or higher than a set height among the three-dimensional point cloud data acquired in the side data acquisition step; a jig detection step of identifying point cloud data on a top surface of a jig placed on the platform using the three-dimensional point cloud data on the front side of the moving body of the detected surface of the luggage among the three-dimensional point cloud data acquired in the side data acquisition step, and further detecting the surface of the jig on the rear side of the moving body using the identified point cloud data; a distance calculation step of calculating a distance between the detected surface of the baggage and the detected surface of the jig; having A method for supporting automated loading, comprising:

6. A method for assisting in automated loading according to claim 5, comprising the steps of: a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving body; a moving object identification step of identifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition step; a platform detection step of identifying a platform height of the moving body using the point cloud data identified by the moving body identification step, and detecting an area on the platform in which the luggage can be loaded using point cloud data that is at or above the identified platform height; Further comprising A method for supporting automated loading, comprising:

7. A method for assisting in automated loading according to claim 6, comprising the steps of: a position of the depth sensor disposed on a side of the moving body is adjusted based on the area in which the luggage can be carried, the position of the depth sensor being adjusted on the side of the moving body, and then the three-dimensional point cloud data is acquired in the side data acquisition step. A method for supporting automated loading, comprising:

8. A method for supporting automated loading according to any one of claims 5 to 7, comprising: The baggage is a cylindrical object, and is placed on the loading platform such that one end surface of the cylinder faces the front side of the moving body and the other end surface faces the rear side of the moving body; The jig is an object having a cube or a rectangular parallelepiped shape, and is placed on the platform so that one surface of the cube or the rectangular parallelepiped faces the front side of the moving body and the opposite surface faces the rear side of the moving body. A method for supporting automated loading, comprising:

9. On the computer, A side data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a baggage detection step of detecting a surface of a baggage placed on a platform of the moving body on a front side of the moving body by using the three-dimensional point cloud data at a height equal to or higher than a set height among the three-dimensional point cloud data acquired in the side data acquisition step; a jig detection step of identifying point cloud data on a top surface of a jig placed on the platform using the three-dimensional point cloud data on the front side of the moving body of the detected surface of the luggage among the three-dimensional point cloud data acquired in the side data acquisition step, and further detecting the surface of the jig on the rear side of the moving body using the identified point cloud data; a distance calculation step of calculating a distance between the detected surface of the baggage and the detected surface of the jig; A program to execute.

10. The program according to claim 9, The computer includes: a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving body; a moving object identification step of identifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition step; a platform detection step of identifying a platform height of the moving body using the point cloud data identified by the moving body identification step, and detecting an area on the platform in which the luggage can be loaded using point cloud data that is at or above the identified platform height; Further execute A program characterized by:

11. The program according to claim 10, a position of the depth sensor disposed on a side of the moving body is adjusted based on the area in which the luggage can be carried, the position of the depth sensor being adjusted on the side of the moving body, and then the three-dimensional point cloud data is acquired in the side data acquisition step. A program characterized by:

12. The program according to any one of claims 9 to 11, The baggage is a cylindrical object, and is placed on the loading platform such that one end surface of the cylinder faces the front side of the moving body and the other end surface faces the rear side of the moving body; The jig is an object having a cube or a rectangular parallelepiped shape, and is placed on the platform so that one surface of the cube or the rectangular parallelepiped faces the front side of the moving body and the opposite surface faces the rear side of the moving body. A program characterized by:

Images