Unloading automation support device, unloading automation support method, and program

Abstract

To assist unloading automation carried out by a transportation system.SOLUTION: An unloading automation assistance device 10 comprises: a rear data acquisition unit 11 that acquires three-dimensional point group data generated from output data of a depth sensor disposed behind a mobile body; a mobile body specification unit 12 that specifies three-dimensional point group data equivalent to the mobile body from the acquired three-dimensional point group data; and a load detection unit 13 that specifies the height of a load carrying platform of the mobile body by using the specified three-dimensional point group data, and further detects the outer edge of the surface of the mobile body rear side of a load disposed on the load carrying platform by using three-dimensional point group data of a load carrying platform height or more.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an unloading automation support device and an unloading automation support method that support unloading in an automatic cargo transport system, and further to a program for realizing these. [Background technology]

[0002] Generally, cranes are used in factories to transport heavy loads, such as coils of thin metal, machine parts, and raw materials. Crane operation is often performed manually, but automation of cranes has been proposed to improve production efficiency.

[0003] For example, Patent Document 1 proposes a crane system that automatically transports cargo. Specifically, the crane system disclosed in Patent Document 1 is composed of a sensor network terminal installed on the cargo, a base station terminal installed on the crane, and a control device.

[0004] The control device first receives the location information of the load's origin and destination from the operator, and then inputs the location information to the crane, instructing it to transport the load.

[0005] Then, when the crane notifies the controller that the transport is complete, the controller identifies the location of each base station (crane) based on the location information of each of the multiple base station terminals. Furthermore, the controller identifies the location of the sensor network terminal, i.e., the location of the cargo, based on the reception time of each base station terminal of the wireless packet transmitted by the sensor network terminal and the location information of each base station terminal. The controller then determines whether the identified location of the cargo matches the location indicated by the input destination location information, and if they match, determines that the transport is complete. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-001804 Summary of the Invention [Problem to be solved by the invention]

[0007] In this way, the crane system disclosed in Patent Document 1 can transport cargo to a specified location without the need for an operator to operate the crane. However, the crane system disclosed in Patent Document 1 has a problem in that it cannot adjust the position of the crane depending on the conditions at the destination.

[0008] For example, consider a case where cargo loaded on a truck bed is unloaded and transported to a designated location within a factory. In this case, the position of the truck is likely to vary, and the truck bed size differs depending on the vehicle model. Therefore, in order to unload the cargo from the truck bed, it is necessary to adjust the position of the crane to match the position of the bed. In addition, a jig is usually installed on the truck bed to secure the cargo during transportation, so the position of the crane also needs to be adjusted depending on the position of the jig. Furthermore, the position of the jig may differ depending on the truck. For this reason, it is difficult to unload cargo from a transport vehicle such as a truck using the crane system disclosed in Patent Document 1.

[0009] An example of an object of the present invention is to provide an unloading automation support device, an unloading automation support method, and a program that can support the automation of unloading by a transport system. [Means for solving the problem]

[0010] In order to achieve the above object, an unloading automation support device according to one aspect of the present invention comprises: a rear data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification unit that identifies three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition unit; a luggage detection unit that uses the 3D point cloud data identified by the moving body identification unit to identify the height of the platform of the moving body, and further uses 3D point cloud data that is at or above the identified height of the platform to detect the surface of the luggage placed on the platform on the rear side of the moving body and the outer edge of the surface; The present invention is characterized in that it is provided with:

[0011] In order to achieve the above object, a method for supporting automation of unloading according to one aspect of the present invention comprises: a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification step of identifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired in the rear data acquisition step; a luggage detection step of identifying a height of a platform of the moving body using the 3D point cloud data identified in the moving body identification step, and further detecting a rearward surface of luggage placed on the platform and an outer edge of the surface using 3D point cloud data at or above the identified height of the platform; The present invention is characterized by having the following:

[0012] Furthermore, in order to achieve the above object, a program according to one aspect of the present invention comprises: On the computer, a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification step of identifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired in the rear data acquisition step; a luggage detection step of identifying a height of a platform of the moving body using the 3D point cloud data identified in the moving body identification step, and further detecting a rearward surface of luggage placed on the platform and an outer edge of the surface using 3D point cloud data at or above the identified height of the platform; The method is characterized in that: [Effects of the Invention]

[0013] As described above, the present invention can assist in automating unloading using a transport system. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an unloading automation support device according to the first embodiment. [Figure 2] 2A and 2B are diagrams showing an example of luggage and moving objects to be unloaded in the first embodiment, where FIG. 2A is a side view and FIG. 2B is a top view. [Figure 3] FIG. 3 is a diagram showing an example of the configuration of the unloading automation support device according to the first embodiment and a transport system to be supported. [Figure 4] Figure 4 shows an example of 3D point cloud data acquired from the rear in embodiment 1, where Figure 4(a) shows the entire 3D point cloud data acquired from the rear depth sensor from diagonally behind the depth sensor, and Figure 4(b) shows the 3D point cloud data corresponding to a truck from the top side of the truck. [Figure 5] Figure 5 is a diagram for explaining the processing in the luggage detection unit in embodiment 1, where Figure 5(a) shows 3D point cloud data near the truck bed from the rear side of the truck, and Figure 5(b) shows 3D point cloud data extracted from the 3D point cloud data shown in Figure 5(a) from the top side of the truck. [Figure 6] FIG. 6 shows the coil material on the loading platform from the side of the truck. [Figure 7] FIG. 7 shows the coil material on the loading platform from the rear side of the truck. [Figure 8] FIG. 8 is a flow diagram showing the operation of the unloading automation support device in the first embodiment. [Figure 9]9A and 9B are diagrams showing an example of luggage and moving objects to be unloaded in the second embodiment, where FIG. 9A is a side view and FIG. 9B is a top view. [Figure 10] FIG. 10 is a diagram showing an example of the configuration of an unloading automation support device according to the second embodiment and a transport system to be supported. [Figure 11] FIG. 11 is a diagram showing an example of three-dimensional point cloud data acquired from the side of a truck in the second embodiment. [Figure 12] Figure 12 is a diagram for explaining the processing in the surface detection unit in embodiment 2, where Figure 12(a) shows three-dimensional point cloud data near the coil material from the side of the track, Figure 12(b) shows the point cloud near the coil material projected onto the XY plane, and Figure 12(c) shows the detected surface of the coil material. [Figure 13] FIG. 13 is a diagram for explaining the processing in the distance calculation unit according to the second embodiment. [Figure 14] FIG. 14 is a flow chart showing the operation of the unloading automation support device in the second embodiment. [Figure 15] FIG. 15 is a block diagram showing an example of a computer that realizes the unloading automation support device according to the first and second embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0015] (Embodiment 1) Hereinafter, the unloading automation support device, the unloading automation support method, and the program according to the first embodiment will be described with reference to FIGS.

[0016] [Device configuration] First, the schematic configuration of the unloading automation support device in the first embodiment will be described with reference to Fig. 1. Fig. 1 is a configuration diagram showing the schematic configuration of the unloading automation support device in the first embodiment.

[0017] The unloading automation support device 10 in the first embodiment shown in Fig. 1 is a device for supporting unloading by a luggage transport system. As shown in Fig. 1, the unloading automation support device 10 includes a rear data acquisition unit 11, a moving object identification unit 12, and a luggage detection unit 13.

[0018] The rear data acquisition unit 11 acquires 3D point cloud data generated from output data of the depth sensor. This depth sensor is disposed behind the moving object and will be referred to as the "rear depth sensor" hereinafter. The moving object identification unit 12 identifies point cloud data corresponding to the moving object from the 3D point cloud data acquired by the rear data acquisition unit 11.

[0019] The luggage detection unit 13 uses the point cloud data identified by the mobile body identification unit 12 to identify the height of the platform of the mobile body, and further uses three-dimensional point cloud data that is at or above the identified platform height to detect the rear surface of the mobile body of the luggage placed on the platform and the outer edge of this surface.

[0020] As described above, the unloading automation support device 10 can detect the rear surface of the moving object of the luggage on the platform of the moving object and the outer edge of that surface. The surface and outer edge are calculated using 3D point cloud data obtained from the depth sensor and are detected with high accuracy. Therefore, the unloading automation support device 10 can support the automation of unloading in a transportation system.

[0021] Next, the luggage and moving objects to be unloaded in the first embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of luggage and moving objects to be unloaded in the first embodiment, Fig. 2(a) is a side view, and Fig. 2(b) is a top view.

[0022] 2(a) and 2(b), in the first embodiment, the cargo 20 is a cylindrical object, specifically, a coil material obtained by rolling a steel plate into a roll. Hereinafter, this will also be referred to as "coil material 20." A cubic or rectangular parallelepiped object is used as a jig 21 for placing the coil material 20.

[0023] In the first embodiment, the moving body 40 is a truck. Hereinafter, this will also be referred to as "truck 40." The truck 40 has a loading platform 41. In the first embodiment, the front side of the moving body will also be referred to as the truck front side, and the rear side of the moving body will also be referred to as the truck rear side.

[0024] The coil material 20 and the jig 21 are placed on the loading platform 41 of the truck 40. At this time, the coil material 20 is placed so that one end face constituting the cylinder faces the front side of the truck and the other end face faces the rear side of the truck. The jig 21 is placed so that one face faces the front side of the truck and the opposite face faces the rear side of the truck. In the example of Figures 2(a) and (b), a skid 22 for fixing the coil material is also placed on the loading platform 41. The skid is made up of two rail-shaped members.

[0025] Furthermore, as shown in FIGS. 2(a) and 2(b), in the first embodiment, a rear depth sensor 31 is installed at the rear of the truck 40. Examples of the rear depth sensor 31 include a time-of-flight (TOF) camera and a light detection and ranging (LiDAR). In the example of FIGS. 2(a) and 2(b), the rear depth sensor 31 is a three-dimensional LiDAR that outputs three-dimensional point cloud data of the subject. Furthermore, as shown in FIG. 3 (described later), the rear depth sensor 31 is connected to the unloading automation support device 10 in the first embodiment, and sends three-dimensional point cloud data from the rear of the truck to the unloading automation support device 10.

[0026] Next, the configuration of the unloading automation support device 10 in the first embodiment will be specifically described with reference to Fig. 3. Fig. 3 is a diagram showing the configuration of the unloading automation support device in the first embodiment and an example of a transport system to be supported. As shown in Fig. 3, the transport system 50 includes a crane 51 installed in a factory and a control device 52.

[0027] In the example of Fig. 3, the crane 51 is an overhead crane that moves along a runway installed near the ceiling of a factory. The type of crane 51 is not particularly limited in the first embodiment. The control device 52 controls the movement of the crane 51 itself, the movement of the hoist, and the up and down movement of the hook. For example, when the position of a load and the position to which the load is to be delivered are specified, the control device 52 controls the crane 51 to deliver the load from the specified position to the destination position.

[0028] 3, the unloading automation support device 10 is connected to the control device 52 so as to be able to communicate data, and transmits information identifying the surface and the outer edge of the detected coil material 20. The control device 52 uses the transmitted information to control the crane 51.

[0029] Next, the functions of the unloading automation support device 10 in the first embodiment will be specifically described with reference to FIGS.

[0030] Figure 4 shows an example of 3D point cloud data acquired from the rear in embodiment 1, where Figure 4(a) shows the entire 3D point cloud data acquired from the rear depth sensor from diagonally behind the depth sensor, and Figure 4(b) shows the 3D point cloud data corresponding to a truck from the top side of the truck.

[0031] In the first embodiment, the rear data acquisition unit 11 acquires three-dimensional point cloud data from a rear depth sensor 31 arranged behind the moving body 40. An example of the acquired three-dimensional point cloud data is shown in FIG. 4(a). In the first embodiment, in the three-dimensional point cloud data, the front-to-rear direction of the truck 40 (see FIG. 2) is defined as the "depth direction: Y direction," the width direction of the truck 40 is defined as the "lateral direction: X direction," and the height direction of the truck 40 is defined as the "height direction: Z direction."

[0032] The moving object identification unit 12 identifies the 3D point cloud data corresponding to the truck 40 shown in Fig. 4(b) from the 3D point cloud data shown in Fig. 4(a). Specifically, the moving object identification unit 12 extracts the 3D point cloud data existing in a predetermined area. The area to be extracted is appropriately set depending on the size of the truck 40, the location of the parking area, etc.

[0033] Figure 5 is a diagram for explaining the processing in the luggage detection unit in embodiment 1, where Figure 5(a) shows 3D point cloud data near the truck bed from the rear side of the truck, and Figure 5(b) shows 3D point cloud data extracted from the 3D point cloud data shown in Figure 5(a) from the top side of the truck.

[0034] 5(a), the baggage detection unit 13 first generates a projection histogram in the height direction (Z direction) of the truck 40 (see FIG. 2) using the 3D point cloud data identified by the moving object identification unit 12. Next, based on this projection histogram, the baggage detection unit 13 calculates the height of the loading platform, the bottom end of the object on the loading platform, and the top end of the object on the loading platform.

[0035] Specifically, for example, the baggage detection unit 13 identifies the location with the largest number of votes among all votes in the projection histogram and sets the height of the identified location as the platform height. The baggage detection unit 13 also adds a predetermined value to the set platform height and sets the resulting value as the lower end of the object on the platform, and adds another predetermined value to the set lower end of the object on the platform and sets the resulting value as the upper end of the object on the platform. The predetermined values ​​used in this case are set appropriately depending on the respective heights of the coil material 20, the jig 21, and the skid 22, as well as the thickness of the members placed on the platform 41, etc.

[0036] Next, the baggage detection unit 13 extracts 3D point cloud data between the lower end of the object on the loading platform and the upper end of the object on the loading platform. The extracted 3D point cloud data is as shown in FIG. 5(b).

[0037] 5(a) and 5(b), the baggage detection unit 13 converts the extracted 3D point cloud data into voxels. The voxelization is performed by, for example, setting one side of each box to 1 cm and eliminating boxes with four or fewer points as noise. Boxes that have not been eliminated as noise are considered valid boxes.

[0038] 6 and 7 are diagrams for explaining the process of detecting the surface and outer edge of the coil material on the bed in embodiment 1. Fig. 6 shows the coil material on the bed from the side of the truck, and Fig. 7 shows the coil material on the bed from the rear of the truck.

[0039] After voxelizing the 3D point cloud data between the bottom and top edges of the object on the loading platform, the baggage detection unit 13 uses this 3D point cloud data to generate a projection histogram (hereinafter referred to as a "Y histogram") in the front-to-back direction (Y direction) of the truck, as shown in Fig. 6. Then, the baggage detection unit 13 searches the Y histogram from the rear side to identify the point where the value on the front side is lower than the value on the rear side and is located closest to the front side.

[0040] Next, as shown in Fig. 6, the baggage detection unit 13 sets a search range to include the identified location, and again searches the Y histogram from the rear side within the set search range. The width of the search range is set in advance. The baggage detection unit 13 then identifies a position where the value is lower than the average value within the search range, and sets the identified position as the position of the surface of the coil material 20 on the rear side of the track.

[0041] Furthermore, the baggage detection unit 13 generates a projection histogram in the width direction (X direction) of the truck and a projection histogram in the height direction (Z direction) of the truck using 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform, as shown in Fig. 7. Hereinafter, the former will be referred to as the "X histogram" and the latter will be referred to as the "Z histogram."

[0042] Next, the baggage detection unit 13 searches each of the X histogram and the Z histogram to set a rectangle (dashed line) surrounding the surface of the coil material, as shown in Fig. 7. Specifically, the baggage detection unit 13 identifies a location where the value is greater than a value obtained by multiplying the maximum value of the histogram by a predetermined coefficient, for example, and sets the rectangle so that the identified location is included in the side of the rectangle.

[0043] Next, the baggage detection unit 13 sets search range 1 to include the right side of the rectangle, and sets search range 2 to include the left side of the rectangle, as shown in Fig. 7. The widths of search ranges 1 and 2 are set in advance. Then, the baggage detection unit 13 searches from the outside of each of search ranges 1 and 2, identifies positions where the value is greater than 0 (zero), and sets the identified positions as the right and left ends of the coil material 20.

[0044] 7, the baggage detection unit 13 also sets a search range 3 so that the search range 3 includes the upper side of the rectangle. The width of the search range 3 is also set in advance. The baggage detection unit 13 then searches within the search range 3 from the top, identifies a position where the value is greater than 0 (zero), and sets the identified position as the top end of the coil material 20. After setting the top end, the baggage detection unit 13 further calculates the distance between the right end and the left end, identifies a position that is the calculated distance below the top end, and sets the identified position as the bottom end.

[0045] In this way, the luggage detection unit 13 detects the positions of the surface on the rear side of the truck, the top end, the bottom end, the right end, and the left end of the coil material 20. The luggage detection unit 13 then transmits information specifying each of the detected positions to the control device 52. As a result, the control device 52 uses the transmitted information to control the crane 51.

[0046] [Device operation] Next, the operation of unloading automation support device 10 in embodiment 1 will be described with reference to Figure 8. Figure 8 is a flow diagram showing the operation of unloading automation support device 10 in embodiment 1. In the following description, Figures 1 to 7 will be referenced as appropriate. In embodiment 1, an unloading automation support method is implemented by operating unloading automation support device 10. Therefore, the description of the unloading automation support method in embodiment 1 will be replaced by the following description of the operation of unloading automation support device 10.

[0047] As shown in FIG. 8, first, the rear data acquisition unit 11 acquires three-dimensional point cloud data from the rear depth sensor 31 arranged at the rear of the truck 40 (step A1).

[0048] Next, the moving object identifying unit 12 identifies point cloud data corresponding to the truck 40 from the three-dimensional point cloud data acquired in step A1 (step A2). Specifically, in step A2, the moving object identifying unit 12 extracts three-dimensional point cloud data present in a predetermined area.

[0049] Next, the luggage detection unit 13 uses the point cloud data identified in step A2 to identify the height of the bed 41 of the truck 40, and further uses three-dimensional point cloud data that is above the identified bed height to detect the surface of the coil material 20 placed on the bed 41 on the rear side of the truck and the outer edge of this surface (step A3).

[0050] Specifically, in step A3, as shown in Fig. 5(a), the baggage detection unit 13 generates a projection histogram in the height direction (Z direction) of the truck 40 (see Fig. 2) using the 3D point cloud data identified in step A2. Furthermore, based on the generated projection histogram, the baggage detection unit 13 calculates the height of the loading platform, the bottom end of the object on the loading platform, and the top end of the object on the loading platform, and identifies the 3D point cloud data located between the bottom end of the object on the loading platform and the top end of the object on the loading platform (see Fig. 5(b)).

[0051] Also, in step A3, the luggage detection unit 13 generates a Y histogram using 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform, as shown in Figure 6, and uses the Y histogram to detect the position of the surface of the coil material 20 on the rear side of the truck.

[0052] Furthermore, in step A3, the baggage detection unit 13 generates an X histogram and a Z histogram using the 3D point cloud data between the bottom end of the object on the loading platform and the top end of the object on the loading platform, as shown in Fig. 7. Then, the baggage detection unit 13 detects the positions of the right end, left end, top end, and bottom end of the coil material 20 using the X histogram and the Z histogram.

[0053] Thereafter, the luggage detection unit 13 transmits to the control device 52 information identifying the surface and outer edge detected in step A3, i.e., information identifying the positions of the surface of the coil material 20 on the rear side of the truck, the top end of the surface, the right end of the surface, and the left end of the surface (step A4).

[0054] As described above, the unloading automation support device 10 detects the surface of the coil material 20 on the rear side of the truck and its outer edges (upper end, lower end, right end, and left end), and transmits information identifying these to the control device 52. This allows the control device 52 to use the transmitted information to control the crane 51.

[0055] [program] The program in the first embodiment may be any program that causes a computer to execute steps A1 to A4 shown in Fig. 8. By installing and executing this program on a computer, the unloading automation support device 10 and the unloading automation support method in the first embodiment can be realized. In this case, the processor of the computer functions and performs processing as a rear data acquisition unit 11, a moving object identification unit 12, and a baggage detection unit 13. Examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0056] The program in the first embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the rear data acquisition unit 11, the moving object identification unit 12, and the baggage detection unit 13.

[0057] (Embodiment 2) Next, an unloading automation support device, an unloading automation support method, and a program according to the second embodiment will be described with reference to FIGS.

[0058] [Device configuration] First, the configuration of the unloading automation support device in the second embodiment will be described with reference to Fig. 9 and Fig. 10. Fig. 9 shows an example of luggage and a moving object to be unloaded in the second embodiment, with Fig. 9(a) being a side view and Fig. 9(b) being a top view. Fig. 10 shows an example of the configuration of the unloading automation support device in the second embodiment and a transport system to be supported.

[0059] As shown in Figures 9(a) and (b), in the second embodiment, the cargo 20 is a coil material, as in the first embodiment. A cubic or rectangular parallelepiped object is used as the jig 21. The jig 21 is arranged so that one surface faces the front side of the moving body and the opposite surface faces the rear side of the moving body. Furthermore, in the second embodiment, the moving body 40 is a truck, as in the first embodiment.

[0060] 9(a) and 9(b), in the second embodiment, similarly to the first embodiment, a rear depth sensor 31 is installed at the rear of the truck 40. The rear depth sensor 31 senses the coil material placed on the loading platform 41 from the rear of the truck.

[0061] However, in the second embodiment, unlike the first embodiment, depth sensors 32 are also installed on the sides of the truck 40. As the depth sensors (hereinafter referred to as "side depth sensors") 32, a three-dimensional LiDAR is used, similar to the rear depth sensor 31. The side depth sensors 32 output three-dimensional point cloud data obtained by sensing from the sides of the truck.

[0062] Furthermore, as shown in FIG. 10 described later, the side depth sensor 32 is connected to an unloading automation support device 60 in the second embodiment, and sends three-dimensional point cloud data from the side of the truck to the unloading automation support device 60.

[0063] Furthermore, the position of the side depth sensor 32 is adjusted by a position adjustment device (see FIG. 10) described below so that it can sense the coil material according to the position of the coil material 20 placed on the loading platform 41. In the example of FIG. 9, the side depth sensor 32 is placed at two positions by the position adjustment device: position A, which faces the surface of the coil material 20 on the front side of the truck, and position B, which faces the surface of the coil material on the rear side of the truck.

[0064] As shown in Fig. 10, the unloading automation support device 60 in the second embodiment is a device for supporting unloading in the transport system 50, similar to the first embodiment. In the second embodiment, the transport system 50 includes a crane 51 installed in a factory, a control device 52, and a position adjustment device 53. The position adjustment device 53 is a device that adjusts the position of the side depth sensor 32 arranged on the side of the truck. Although not shown in Fig. 10, the position adjustment device 53 includes a movement mechanism such as a ball screw, a driving electric motor, etc.

[0065] As in the first embodiment, the crane 51 is an overhead crane that moves along a runway installed near the ceiling of the factory. As in the first embodiment, the control device 52 is connected to the unloading automation support device 60 so as to be able to communicate data with it. The control device 52 uses information transmitted from the unloading automation support device 60 to control the crane 51.

[0066] In the second embodiment, the control device 52 sets, for example, positions A and B shown in FIG. 9 based on the information on the surface and outer edge of the coil material 20 (see the first embodiment) transmitted from the unloading automation support device 60. Then, the control device 52 controls the position adjustment device 53 to set the position of the side depth sensor 32 to position A, and then sets the position of the side depth sensor 32 to position B.

[0067] As shown in Figure 10, the unloading automation support device 60 includes a rear data acquisition unit 11, a moving object identification unit 12, a luggage detection unit 13, a side data acquisition unit 14, a surface detection unit 15, and a distance calculation unit 16.

[0068] Of these, the rear data acquisition unit 11, the moving object identification unit 12, and the baggage detection unit 13 are configured in the same manner as in embodiment 1, and further have the same functions. Therefore, in embodiment 2, the description of the rear data acquisition unit 11, the moving object identification unit 12, and the baggage detection unit 13 will be omitted.

[0069] The side data acquisition unit 14 acquires three-dimensional point cloud data from side depth sensors 32 arranged on the sides of the truck 40. The surface detection unit 15 detects the surface of the coil material 20 on the front side of the track and the surface of the coil material 20 on the rear side of the track using the three-dimensional point cloud data acquired by the side data acquisition unit 14. The distance calculation unit 16 calculates the distance between the surface of the coil material 20 on the front side of the track and the surface of the coil material 20 on the rear side of the track detected by the surface detection unit 15.

[0070] 11 to 13, the functions of unloading automation support device 60 in the second embodiment will be specifically described. Fig. 11 is a diagram showing an example of three-dimensional point cloud data acquired from the side in the second embodiment, viewed from the side of a truck.

[0071] In the second embodiment, the lateral depth sensor 32 performs sensing at two positions, position A and position B, shown in Fig. 10. Therefore, the lateral data acquisition unit 14 acquires three-dimensional point cloud data at position A and three-dimensional point cloud data at position B.

[0072] An example of the three-dimensional point cloud data acquired at this time is shown in the upper diagram of Fig. 10. The example of Fig. 10 shows three-dimensional point cloud data at position A. Furthermore, in the second embodiment, in the three-dimensional point cloud data, the front-to-rear direction of the track 40 (see Fig. 9) is defined as the "lateral direction: Y direction", the width direction of the track 40 is defined as the "depth direction: X direction", and the height direction of the track 40 is defined as the "height direction: Z direction".

[0073] Figure 12 is a diagram for explaining the processing in the surface detection unit in embodiment 2, where Figure 12(a) shows three-dimensional point cloud data near the coil material from the side of the track, Figure 12(b) shows the point cloud near the coil material projected onto the XY plane, and Figure 12(c) shows the detected surface of the coil material.

[0074] The surface detection unit 15 first extracts 3D point cloud data in the near distance region from the 3D point cloud data acquired by the side data acquisition unit 14. Specifically, as shown in FIG. 11, the surface detection unit 15 extracts 3D point cloud data consisting only of points whose depth values ​​are equal to or less than a set value. This removes point clouds of objects located outside the bed of the truck 40. The region surrounded by a dashed frame in the upper diagram of FIG. 11 is the extracted region. The lower diagram of FIG. 11 shows only the extracted 3D point cloud data.

[0075] Next, the surface detection unit 15 converts the extracted 3D point cloud data into voxels, although this is not shown in Fig. 11. As described in the baggage detection unit 13, the voxelization is performed by, for example, setting one side of each box to 1 cm and eliminating boxes with four or fewer points as noise. Boxes that have not been eliminated as noise are considered valid boxes.

[0076] Furthermore, as shown in Fig. 12(a), the surface detection unit 15 identifies the 3D point cloud data of the upper half (the portion surrounded by a dashed frame line) of the effective box of the 3D point cloud data extracted in Fig. 11. Then, as shown in Fig. 12(b), the surface detection unit 15 projects the identified 3D point cloud data onto the XY plane.

[0077] Next, the surface detection unit 15 performs a Hough transform on the point cloud projected onto the XY plane shown in FIG. 12(b) to detect multiple candidates for line segments on the XY plane that may be surfaces of the coil material 20 (see FIG. 9). Next, the surface detection unit 15 determines a line segment corresponding to the surface of the coil material 20 from the multiple candidates in accordance with set conditions. Then, the surface detection unit 15 detects a surface that includes the determined line segment and is perpendicular to the XY plane as the surface of the coil material 20. The set conditions include a small angle with respect to an axis along the Y direction, a long straight portion, etc.

[0078] The above-described processing by the surface detection unit 15 is performed for each of the three-dimensional point cloud data acquired for each position. As a result, the surface of the coil material 20 on the front side of the track is detected from the three-dimensional point cloud data acquired at position A, and the surface of the coil material 20 on the rear side of the track is detected from the three-dimensional point cloud data acquired at position B.

[0079] Fig. 13 is a diagram for explaining the processing in the distance calculation unit in the second embodiment. In the second embodiment, as shown in Fig. 13, the distance calculation unit 16 calculates the distance between the surface of the coil material 20 on the front side of the track and the surface of the coil material 20 on the rear side of the track, which are detected by the surface detection unit 15. The calculated distance corresponds to the width of the coil material 20. Then, the distance calculation unit 16 transmits information specifying the calculated width of the coil material 20 to the control device 52. The control device 52 then uses the transmitted information to control the crane 51.

[0080] [Device operation] Next, the operation of unloading automation support device 60 in embodiment 2 will be described with reference to Figure 14. Figure 14 is a flow diagram showing the operation of unloading automation support device 60 in embodiment 2. In the following description, Figures 9 to 13 will be referenced as appropriate. Also, in embodiment 2, an unloading automation support method is implemented by operating unloading automation support device 60. Therefore, the description of the unloading automation support method in embodiment 2 will be replaced by the following description of the operation of unloading automation support device 60.

[0081] 14, first, the rear data acquisition unit 11 acquires three-dimensional point cloud data from the rear depth sensor 31 arranged at the rear of the truck 40 (step B1). Step B1 is the same step as step A1 shown in FIG.

[0082] Next, the moving object specifying unit 12 specifies point cloud data corresponding to the truck 40 from the three-dimensional point cloud data acquired in step B1 (step B2). Step B2 is the same step as step A2 shown in FIG.

[0083] Next, the luggage detection unit 13 uses the point cloud data identified in step B2 to identify the height of the bed 41 of the truck 40, and further uses the 3D point cloud data that is at or above the identified height of the bed to detect the surface of the coil material 20 placed on the bed 41 on the rear side of the truck and the outer edge of this surface (step B3). Step B3 is a step similar to step A3 shown in FIG. 8.

[0084] Next, the baggage detection unit 13 transmits information specifying the surface and outer edge detected in step B3 to the control device 52 (step B4). Step B4 is the same step as step A4 shown in FIG.

[0085] When step B4 is executed, in the transportation system 50, the control device 52 controls the position adjustment device 53 based on the transmitted information to adjust the position of the side depth sensor 32 on the side of the truck 40. Specifically, the control device 52 sets positions A and B, for example, as shown in FIG. 9, based on the information transmitted from the automated unloading support device 60. Then, the control device 52 controls the position adjustment device 53 to set the position of the side depth sensor 32 to position A, and then to position B. The side depth sensor 52 then performs sensing for each position and outputs 3D point cloud data.

[0086] After step B4 is executed, when sensing is performed for each position by the side depth sensor 32, the side data acquisition unit 14 acquires three-dimensional point cloud data for each position from the side depth sensor 32 (step B5).

[0087] Next, the surface detection unit 15 detects the surface of the coil material 20 on the rear side of the track using the three-dimensional point cloud data acquired at position B (step B6). Furthermore, the surface detection unit 15 detects the surface of the coil material 20 on the front side of the track using the three-dimensional point cloud data acquired at position A (step B7).

[0088] Specifically, in steps B6 and B7, the surface detection unit 15 extracts 3D point cloud data that is in close proximity from the acquired 3D point cloud data, as shown in Fig. 11. Furthermore, the surface detection unit 15 identifies 3D point cloud data near the coil material from the extracted 3D point cloud data, and projects this onto the XY plane, as shown in Fig. 12(a).

[0089] Furthermore, in steps B6 and B7, the surface detection unit 15 performs a Hough transform on the point group projected onto the XY plane shown in Fig. 12(b) to detect multiple candidates for line segments on the XY plane that may be surfaces of the coil material 20 (see Fig. 2). Then, the surface detection unit 15 determines the line segment corresponding to the surface of the coil material 20 from the multiple candidates in accordance with the set conditions.

[0090] Next, the distance calculation unit 16 calculates the distance between the surface detected in step B6 and the surface detected in step B7, that is, the width of the coil material 20 (step B8).

[0091] Next, the distance calculation unit 16 transmits information specifying the width of the coil material 20 calculated in step B8 to the control device 52 (step B9). As a result, the control device 52 uses the transmitted information to control the crane 51.

[0092] As described above, the unloading automation support device 60 automatically adjusts the positions of the lateral depth sensors 32 arranged on the sides, and calculates the width of the coil material 20 on the bed of the truck 40 from the 3D point cloud data obtained by sensing after the adjustment. According to the second embodiment, it is possible to further support the automation of unloading of the coil material 20 from the truck 40 by the transport system 50.

[0093] [program] The program in the second embodiment may be a program that causes a computer to execute steps B1 to B9 shown in Fig. 14. By installing and executing this program in a computer, the unloading automation support device 60 and the unloading automation support method in the second embodiment can be realized. In this case, the processor of the computer functions as a rear data acquisition unit 11, a moving object identification unit 12, a baggage detection unit 13, a side data acquisition unit 14, a surface detection unit 15, and a distance calculation unit 16 to perform processing. Examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0094] The program in the second embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the rear data acquisition unit 11, the moving object identification unit 12, the baggage detection unit 13, the side data acquisition unit 14, the plane detection unit 15, and the distance calculation unit 16.

[0095] (physical configuration) Here, a computer that realizes the unloading automation support device by executing the programs in the first and second embodiments will be described with reference to Fig. 15. Fig. 15 is a block diagram showing an example of a computer that realizes the unloading automation support device in the first and second embodiments.

[0096] 15, a computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0097] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0098] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0099] The programs in the first and second embodiments are provided in a state stored in a computer-readable recording medium 120. The programs in the first and second embodiments may be distributed over the Internet connected via the communication interface 117.

[0100] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0101] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0102] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0103] The unloading automation support device in the embodiment can be realized by using hardware corresponding to each part, rather than a computer on which a program is installed. Furthermore, the unloading automation support device may be realized in part by a program and in the remaining part by hardware.

[0104] Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 12) described below, but are not limited to the following descriptions.

[0105] (Appendix 1) a rear data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification unit that identifies three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition unit; a luggage detection unit that uses the 3D point cloud data identified by the moving body identification unit to identify the height of the platform of the moving body, and further uses 3D point cloud data that is at or above the identified height of the platform to detect the surface of the luggage placed on the platform on the rear side of the moving body and the outer edge of the surface; Equipped with An automated unloading support device characterized by the above.

[0106] (Appendix 2) 10. The unloading automation support device according to claim 1, a side data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving object; a surface detection unit that detects a surface of the luggage on a front side of the moving body and a surface of the luggage on a rear side of the moving body using the three-dimensional point cloud data acquired by the side data acquisition unit; a distance calculation unit that calculates a distance between a surface of the detected luggage on a front side of the moving body and a surface of the detected luggage on a rear side of the moving body; Further comprising: An automated unloading support device characterized by the above.

[0107] (Appendix 3) 3. The unloading automation support device according to claim 2, the position of the depth sensor disposed on the side of the moving object is adjusted based on the surface and the outer edge detected by the baggage detection unit, and then the side data acquisition unit acquires the three-dimensional point cloud data. An automated unloading support device characterized by the above.

[0108] (Appendix 4) An unloading automation support device according to any one of appendices 1 to 3, The luggage is a cylindrical object and is placed on the loading platform so that one end face of the cylinder faces the front side of the moving body and the other end face faces the rear side of the moving body. An automated unloading support device characterized by the above.

[0109] (Appendix 5) a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification step of identifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired in the rear data acquisition step; a luggage detection step of identifying a height of a platform of the moving body using the 3D point cloud data identified in the moving body identification step, and further detecting a rearward surface of luggage placed on the platform and an outer edge of the surface using 3D point cloud data at or above the identified height of the platform; having A method for supporting automated unloading.

[0110] (Appendix 6) 6. The unloading automation support method according to claim 5, a side data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a surface detection step of detecting a surface of the luggage on a front side of the moving body and a surface of the luggage on a rear side of the moving body using the three-dimensional point cloud data acquired in the side data acquisition step; a distance calculation step of calculating a distance between a surface of the detected luggage on a front side of the moving body and a surface of the detected luggage on a rear side of the moving body; Further comprising: A method for supporting automated unloading.

[0111] (Appendix 7) 7. The unloading automation support method according to claim 6, a position of the depth sensor disposed on a side of the moving body is adjusted based on the surface and the outer edge detected in the baggage detection step, and then the 3D point cloud data is acquired in the side data acquisition step; A method for supporting automated unloading.

[0112] (Appendix 8) The unloading automation support method according to any one of Supplementary Notes 5 to 7, The luggage is a cylindrical object and is placed on the loading platform so that one end face of the cylinder faces the front side of the moving body and the other end face faces the rear side of the moving body. A method for supporting automated unloading.

[0113] (Appendix 9) On the computer, a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification step of identifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired in the rear data acquisition step; a luggage detection step of identifying a height of a platform of the moving body using the 3D point cloud data identified in the moving body identification step, and further detecting a rearward surface of luggage placed on the platform and an outer edge of the surface using 3D point cloud data at or above the identified height of the platform; A program that executes.

[0114] (Appendix 10) 10. The program of claim 9, The computer, a side data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a surface detection step of detecting a surface of the luggage on a front side of the moving body and a surface of the luggage on a rear side of the moving body using the three-dimensional point cloud data acquired in the side data acquisition step; a distance calculation step of calculating a distance between a surface of the detected luggage on a front side of the moving body and a surface of the detected luggage on a rear side of the moving body; Further execute A program characterized by:

[0115] (Appendix 11) 11. The program of claim 10, a position of the depth sensor disposed on a side of the moving body is adjusted based on the surface and the outer edge detected in the baggage detection step, and then the 3D point cloud data is acquired in the side data acquisition step; A program characterized by:

[0116] (Appendix 12) The program according to any one of Supplementary Notes 9 to 11, The luggage is a cylindrical object and is placed on the loading platform so that one end face of the cylinder faces the front side of the moving body and the other end face faces the rear side of the moving body. A program characterized by: [Industrial Applicability]

[0117] As described above, the present invention can support automation of unloading by a transport system. The present invention is useful in a system that promotes automation of cargo transport. [Explanation of symbols]

[0118] 10. Automated unloading support device 11 Rear data acquisition section 12 Mobile Identification Unit 13 Baggage detection unit 14 Lateral data acquisition section 15 Face detection unit 16 Distance calculation unit 20 Luggage (coil material) 21 Jig 22 Skid 31 Depth sensor 32 Depth sensor 40 Mobile (Truck) 41 Cargo bed 50 Conveying System 51 Crane 52 Control device 53 Position adjustment device 60 Unloading automation support device (embodiment 2) 110 Computer 111 CPU 112 main memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Reader / Writer 117 Communication Interface 118 Input Devices 119 Display Device 120 Recording Media 121 Bus

Claims

1. a rear data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object identification unit that identifies three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired by the rear data acquisition unit; a luggage detection unit that uses the three-dimensional point cloud data identified by the moving body identification unit to identify the height of the platform of the moving body, and further uses three-dimensional point cloud data that is at or above the identified height of the platform to detect the surface of the luggage placed on the platform on the rear side of the moving body and the outer edge of the surface; a side data acquisition unit that acquires three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving object; a surface detection unit that detects a surface of the luggage on a front side of the moving body and a surface of the luggage on a rear side of the moving body using the three-dimensional point cloud data acquired by the side data acquisition unit; a distance calculation unit that calculates a distance between a surface of the detected luggage on a front side of the moving body and a surface of the detected luggage on a rear side of the moving body; Equipped with An automated unloading support device characterized by the above.

2. The unloading automation support device according to claim 1, the position of the depth sensor disposed on the side of the moving object is adjusted based on the surface and the outer edge detected by the baggage detection unit, and then the side data acquisition unit acquires the three-dimensional point cloud data. An automated unloading support device characterized by the above.

3. The unloading automation support device according to claim 1, The luggage is a cylindrical object and is placed on the loading platform so that one end face of the cylinder faces the front side of the moving body and the other end face faces the rear side of the moving body. An automated unloading support device characterized by the above.

4. a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object specifying step of specifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired in the rear data acquiring step; a luggage detection step of identifying a height of a platform of the moving body using the three-dimensional point cloud data identified in the moving body identification step, and further detecting a rearward surface of luggage placed on the platform and an outer edge of the surface using three-dimensional point cloud data at or above the identified height of the platform; a side data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a surface detection step of detecting a surface of the luggage on a front side of the moving body and a surface of the luggage on a rear side of the moving body using the three-dimensional point cloud data acquired in the side data acquisition step; a distance calculation step of calculating a distance between a surface of the detected luggage on a front side of the moving body and a surface of the detected luggage on a rear side of the moving body; having A method for supporting automated unloading.

5. 5. The unloading automation support method according to claim 4, a position of the depth sensor disposed on a side of the moving body is adjusted based on the surface and the outer edge detected in the baggage detection step, and then the three-dimensional point cloud data is acquired in the side data acquisition step; A method for supporting automated unloading.

6. 5. The unloading automation support method according to claim 4, The luggage is a cylindrical object and is placed on the loading platform so that one end face of the cylinder faces the front side of the moving body and the other end face faces the rear side of the moving body. A method for supporting automated unloading.

7. On the computer, a rear data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed behind the moving object; a moving object specifying step of specifying three-dimensional point cloud data corresponding to the moving object from the three-dimensional point cloud data acquired in the rear data acquiring step; a luggage detection step of identifying a height of a platform of the moving body using the three-dimensional point cloud data identified in the moving body identification step, and further detecting a rearward surface of luggage placed on the platform and an outer edge of the surface using three-dimensional point cloud data at or above the identified height of the platform; a side data acquisition step of acquiring three-dimensional point cloud data generated from output data of a depth sensor disposed on a side of the moving body; a surface detection step of detecting a surface of the luggage on a front side of the moving body and a surface of the luggage on a rear side of the moving body using the three-dimensional point cloud data acquired in the side data acquisition step; a distance calculation step of calculating a distance between a surface of the detected luggage on a front side of the moving body and a surface of the detected luggage on a rear side of the moving body; A program that executes.

8. 8. The program according to claim 7, a position of the depth sensor disposed on a side of the moving body is adjusted based on the surface and the outer edge detected in the baggage detection step, and then the three-dimensional point cloud data is acquired in the side data acquisition step; A program characterized by:

9. 8. The program according to claim 7, The luggage is a cylindrical object and is placed on the loading platform so that one end face of the cylinder faces the front side of the moving body and the other end face faces the rear side of the moving body. A program characterized by:

Images