Pallet detection apparatus and pallet detection method

Abstract

To provide a pallet detection apparatus configured to quickly and accurately identify the number of pallets placed on a platform, in a simple method.SOLUTION: A pallet detection apparatus 10 is configured to: perform pallet detection on a track platform, using two pieces of image information captured from positions in two directions at different angles; determine highly reliable pallet information in image information obtained by performing projective transformation on the two pieces of image information; and identify the types and the number of the pallets placed on the platform based on the highly reliable pallet information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a pallet detection technique for detecting a pallet placed on a loading platform. [Background technology]

[0002] As a technology for managing cargo in warehouses, a method has been proposed in which the locations of cargo and pallets within the warehouse are managed by reading the identification codes attached to the cargo and pallets.Recently, a technology has been proposed that reduces the workload of inspecting cargo and taking inventory within the warehouse by acquiring image information of the cargo and pallets within the warehouse using cameras mounted on guided vehicles or drones that move within the warehouse.

[0003] For example, Patent Documents 1 and 2 propose a pallet detection system that uses a robot or drone equipped with a camera that takes images of packages in a warehouse. The camera mounted on the robot or drone can read barcodes attached to packages or shelves to obtain information on the type and number of packages. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-527172 [Patent Document 2] Japanese Patent Application Publication No. 2019-167177 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the above-mentioned technology for reading barcodes attached to packages or pallets requires capturing images of the packages or pallets from a relatively short distance and from the front, so when inspecting packages or pallets placed on the back of a truck, etc., it may not be possible to accurately detect the packages or pallets. When packages placed on the back of a truck, etc. are being brought in or taken out, there is a demand for technology that can accurately determine the number of packages or pallets in a simpler, faster, and more accurate way than when carrying out packages placed in a warehouse.

[0006] The present invention has been made to solve the above problems, and aims to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler way. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the pallet detection device of the present invention comprises an imaging unit that captures images of a pallet loaded on a loading platform from different angles to obtain first image information and second image information captured at different imaging angles; a pallet detection unit that detects image information of the pallet from the first image information and the second image information by comparing the first image information and the second image information with pre-registered image information of the pallet, and obtains pallet information including type information, position information, size information, and a reliability coefficient for pallet detection of the detected pallet; an image processing unit that performs projective transformation on the first image information and the second image information, respectively, to obtain third image information and fourth image information having a predetermined shape, and obtains the position information and size information of the pallet transformed as a result of the projective transformation; and a pallet determination unit that determines the pallet information with a high reliability coefficient from the pallet information of pallets loaded at the same position on the loading platform in the third image information and the fourth image information.

[0008] In addition, the pallet determination unit may determine whether the pallet is loaded in the same position on the loading platform based on the difference between the position information and size information of the pallet in the third image information and the position information and size information of the pallet in the fourth image information. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform using a simpler method. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 shows an example of a system configuration including a pallet detection device according to an embodiment of the present invention. [Figure 2] FIG. 2 is an example of a functional block diagram of a pallet detection device according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram for explaining a method for detecting palette information in the embodiment of the present invention. [Figure 4] FIG. 4 is a diagram for explaining the detection result of palette information in the embodiment of the present invention. [Figure 5] FIG. 5 shows an example of an operational flow of the palette detection method according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention can be embodied in various embodiments and is not limited to the embodiments of the invention described below.

[0012] <Configuration of Pallet Detection System> Fig. 1 shows an example of the configuration of a pallet detection system according to an embodiment of the present invention. The pallet detection system shown in Fig. 1 includes a pallet detection device 10 that acquires an image of a pallet 3 on which cargo 2 is placed, placed on the loading platform of a transport vehicle such as a truck 1, and detects the pallet 3 on the loading platform using the acquired image.

[0013] The pallet detection system of Figure 1 is configured to detect a pallet 3 on which cargo 2 is placed that is loaded on the bed of a truck 1 by comparing image information acquired by an imaging unit mounted on the pallet detection device 10 with image information of a pre-registered pallet.

[0014] When loading or unloading cargo 2 placed on the loading platform of a truck 1 or the like, it is necessary to detect the number of pallets 3 placed on the loading platform of the truck 1 or the like to determine whether the cargo to be transported is loaded. Therefore, compared to checking cargo placed in a warehouse, there are time and space constraints, and there is a demand for a simpler method to quickly and accurately determine the number of cargoes and pallets.

[0015] In the configuration example of Figure 1, pallets 3, each holding a plurality of packages 2, are stacked in multiple layers on the loading platform of truck 1. Since the number of pallets 3 on the loading platform of truck 1 is not limited to one type, it is necessary to obtain an image of all the packages on the loading platform of truck 1 in order to ascertain the types and numbers of the multiple types of pallets.

[0016] The pallet detection device 10 acquires image information of the entire loading platform, including an image of the pallet 3, and compares it with pre-stored image information of the pallet to detect the image of the pallet from the image information of the entire loading platform. The pre-stored image information of the pallet includes information representing the characteristics of the pallet, such as the color and shape of the pallet.

[0017] The image information of the pallets that is stored in advance may be images taken for each type of pallet, or image information learned using image information from many pallets. By detecting information that represents the characteristics of the pallets, such as the color and shape of the pallets, from the image information of the entire loading platform, including the image information of pallet 3, it is possible to determine the type and number of pallets in the loading platform of truck 1.

[0018] When automatically detecting the type and number of pallets loaded on a truck bed from images captured by a camera, if an image captured from the front of the truck bed can be obtained, pallets can be detected relatively easily by using image information that has been machine-learned using pre-prepared learning images. However, at truck berths, the shooting space is limited and it is often not possible to ensure a sufficient shooting distance from the truck bed. In such cases, the truck bed must be photographed from an angle to obtain an image of the entire truck bed, which can reduce the detection accuracy of pallets that are far from the shooting position.

[0019] The pallet detection device 10 of this embodiment is configured to take images of the truck bed from two different angle positions, perform projective transformation on the image information taken from the two different angle positions so that it becomes a frontal image, and use the pallet information with the highest reliability for pallet detection at the pallet position detected in the two images after projective transformation as the pallet detection result.

[0020] By detecting pallets using two images taken from two different angles of the truck bed, it is possible to accurately determine the type and number of pallets using the camera images, even in cases where the shooting space is limited, such as a truck berth.

[0021] <Configuration of the pallet detection device> The configuration of the pallet detection device 10 will be described with reference to Fig. 2. Fig. 2 shows an example of functional blocks of the pallet detection device according to the embodiment of the present invention.

[0022] The pallet detection device 10 comprises an imaging unit 11 for acquiring image information of the loading platform including image information of the pallet, a central processing unit 12 for image processing of the image acquired by the imaging unit 11 and for pallet detection and pallet judgment, a memory unit 16 for storing various information, etc., an input unit 17 for the photographer to input information, and a display unit 18 for displaying the detection status of the pallet image.

[0023] The imaging unit 11 is an imaging element for acquiring image information of the entire loading platform, including an image of the pallet 3. A general-purpose RGB camera for detecting the color and shape of the pallet from the image information of the entire loading platform can be used as the imaging unit 11. For example, a digital camera or a smartphone equipped with an RGB camera can be used as the pallet detection device 10 equipped with an imaging unit.

[0024] The central processing unit 12 functions as a palette detection unit 13 that detects palettes from images acquired by the imaging unit 11, an image processing unit 14 that performs image processing such as projective transformation of the acquired images, and a palette determination unit 15 that determines reliable palette information from the detected palette information. The palette detection device 10 can be configured by a computer equipped with a CPU, memory, and an interface for connecting to external devices.

[0025] The storage unit 16 can store pallet information such as pallet image information for comparison with the acquired image information of the loading bay, pallet type information, position information, size information, and pallet detection reliability coefficient. The pallet image information includes information representing the pallet's characteristics such as the color and shape.

[0026] <Detecting pallet information> Fig. 3 is a diagram for explaining a method for detecting pallet information in an embodiment of the present invention. In the example of Fig. 3, image information (A, B, C, D) of the loading platform is acquired from different angles on the left and right of the loading platform, and pallet detection is performed in the acquired image information, and projective transformation of the image information is also performed.

[0027] In Figure 3, image information (A, B, C, D) of the loading platform is obtained from different angles on the left and right sides of the loading platform, and the left image information (first image information) and right image information (second image information) are compared with image information of a pre-registered pallet to detect image information of the pallet from each of the left image information and right image information, and pallet information is obtained including type information, position information, size information, and a reliability coefficient for pallet detection of the detected pallet.

[0028] As shown in Figure 3, it is desirable to acquire image information of the loading platform from angles on the left and right sides of the loading platform from the front direction, particularly when the distance from the loading platform that can be photographed is short, it is expected that the reliability of pallet detection will decrease the farther away from the photographing position, so by acquiring images of the entire loading platform from photographing positions on the left and right sides of the loading platform from the front direction, and performing pallet detection using the two pieces of image information acquired at the left and right photographing positions, it is possible to acquire more reliable pallet information.

[0029] Pallet type information is identified based on pre-registered pallet characteristics such as color and shape. Pallet position information is relative pixel position information of the pallet in the acquired image information, and can be expressed as XY coordinates with A as the origin in the image information (A, B, C, D) of the loading bay in Figure 3. Pallet size information can also be expressed as relative pixel size information in the acquired image information.

[0030] The reliability coefficient of palette detection is a numerical representation of the degree of similarity in terms of "color," "shape," "image contrast," etc., when comparing the acquired image information with the image information of a pre-registered palette. For example, a reliability coefficient of 1 is the most reliable, and a reliability coefficient of 0 is the least reliable.

[0031] When the space available for photographing a loading platform is limited, such as at a truck berth, the platform is photographed from an angle rather than from the front, resulting in distorted image information, as shown in Figure 3. When detecting a pallet image from such image information, it is expected that the reliability of pallet detection will decrease the further away from the photographing position.

[0032] In this embodiment, as shown in Figure 3, images are taken from two positions at different angles relative to the loading platform, and the captured left image information (first image information) and right image information (second image information) are subjected to projective transformation to form a frontal image, thereby obtaining image information (third image information and fourth image information) having a predetermined shape.

[0033] Along with this projective transformation, the position information and size information of the palette detected in the left image information (first image information) and right image information (second image information) are transformed along with the projective transformation of the image information.

[0034] In this embodiment, pallet information with high reliability for pallet detection is determined based on the pallet position in the image information (third image information and fourth image information) corresponding to the left image information and right image information obtained by performing projective transformation, and the type and number of pallets loaded on the loading platform are determined based on the highly reliable pallet information.

[0035] <Determining Pallet Information> Fig. 4 is a diagram for explaining the detection result of palette information in the embodiment of the present invention, which shows palette information at palette positions in the two pieces of image information that have undergone projective transformation in Fig. 3.

[0036] In Figure 4, x and y represent the coordinate position of the palette, w and h represent the size (width and height) of the palette, Δ(x, y) represents the Euclidean distance between the coordinate positions in the left and right image information, and Δ(w, h) represents the area ratio of the palette images in the left and right image information. The palette coordinate position (x, y) and palette size (w, h) are transformed in accordance with the projective transformation of the image information.

[0037] In this embodiment, it is possible to determine that the palettes are in the same palette position based on the difference in the palette coordinate position and palette size between the left and right image information. For example, it may be configured to determine that the palettes are in the same palette position when Δ(x, y) and Δ(w, h) are equal to or smaller than a predetermined threshold.

[0038] In this embodiment, the pallet reliability coefficients of the pallet information at the same pallet position in Figure 4 are compared to determine the pallet information with the highest reliability coefficient, and the type and number of pallets loaded on the loading platform are determined based on the pallet information with the highest reliability coefficient. For example, at pallet position [X0, Y0], the pallet information in the left image with the highest reliability coefficient is used. The same applies to other pallet positions. In the example of Figure 4, two pallets with the pallet type "green," six pallets with the pallet type "orange," and two pallets with the pallet type "blue" are detected.

[0039] <Operation flow of the pallet detection device> The palette detection operation in palette detection device 10 will be described with reference to Fig. 5. Fig. 5 shows an example of the operation flow of the palette detection method according to the embodiment of the present invention.

[0040] The imaging unit 11 of the pallet detection device 10 captures images of the pallet loaded on the loading platform from different angles, thereby acquiring left image information and right image information captured at different angles (S1-1).

[0041] The palette detection unit 13 of the palette detection device 10 compares the left and right image information with the image information of a palette registered in advance, thereby detecting the palette image information from each of the left and right image information, and obtaining palette information including the type information, position information, size information, and palette detection reliability coefficient of the detected palette (S1-2).

[0042] The image processing unit 14 of the palette detection device 10 performs projective transformation on the image information of each of the left image information and the right image information to obtain image information having a predetermined shape, and obtains the position information and size information of the palette transformed in accordance with the projective transformation (S1-3).

[0043] The palette determination unit 15 of the palette detection device 10 aligns the projectively transformed image information and determines palette information with a high reliability coefficient from the palette information loaded at the same palette position in the projectively transformed image information (S1-4, S1-5).

[0044] According to this embodiment, pallets are detected using two sets of image information taken from two different angled positions on the truck bed, and pallet information with a high reliability coefficient is determined from the image information obtained by performing a projection transformation on the two sets of image information.The type and number of pallets loaded on the bed are determined based on the pallet information with the high reliability coefficient.Therefore, even in cases where the space for photography is limited, such as a truck berth, it is possible to accurately detect the type and number of pallets loaded on the bed using camera-captured images. [Explanation of symbols]

[0045] 1...truck, 2...baggage, 3...pallet, 10...pallet detection device, 11...imaging unit, 12...central processing unit, 13...pallet detection unit, 14...image processing unit, 15...pallet determination unit, 16...memory unit, 17...input unit, 18...display unit.

Claims

1. an imaging unit that captures images of the pallet loaded on the loading platform from different angles to obtain first image information and second image information captured at different imaging angles; a palette detection unit that detects image information of the palette from the first image information and the second image information by comparing the first image information and the second image information with image information of the palette that has been registered in advance, and acquires palette information including type information, position information, size information, and a palette detection reliability coefficient of the detected palette; an image processing unit that performs projective transformation on the first image information and the second image information to obtain third image information and fourth image information having predetermined shapes, and obtains the position information and the size information of the palette transformed in accordance with the projective transformation; a pallet determination unit that determines, from the third image information and the fourth image information, the pallet information of the pallets loaded at the same position on the loading platform, the pallet information having a high reliability coefficient; A pallet detection device comprising:

2. The pallet determination unit determines whether the pallets are loaded at the same position on the loading platform based on a difference between the position information and the size information of the pallet in the third image information and the position information and the size information of the pallet in the fourth image information.

2. The pallet detection device according to claim 1.

Images