Palette Detection Device and Palette Detection Method

Abstract

Provided is a pallet detection device that can quickly and accurately grasp the number of pallets installed on a loading platform in a simpler way. 【Solution means】The pallet detection device 10 of the present invention performs pallet detection using two pieces of image information taken from two positions in two directions with different angles with respect to the truck loading platform, determines highly reliable pallet information in the image information obtained by performing projective transformation on the two pieces of image information, and grasps the type and number of pallets loaded on the loading platform based on the highly reliable pallet information.

Description

Technical Field

[0001] The present invention relates to a pallet detection technique for detecting a pallet installed on a loading platform.

Background Art

[0002] As a technique for managing goods in a warehouse, a method has been proposed for managing the installation locations of goods and pallets in the warehouse by reading identification codes assigned to the goods and pallets. Recently, techniques have been proposed for reducing the load of inspections and inventory work of goods in a warehouse by acquiring image information of goods and pallets in the warehouse using cameras mounted on transport vehicles or drones moving in the warehouse.

[0003] For example, in Patent Documents 1 and 2, a pallet detection system using a robot or drone equipped with a camera for photographing an image of goods in a warehouse has been proposed. By reading barcodes installed on goods or shelves with a camera mounted on a robot or drone, information on the types and quantities of goods can be obtained.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, in the technology of reading barcodes installed on packages or pallets as described above, since it is necessary to acquire images of packages or pallets from a relatively short distance and in the front direction, when inspecting packages or pallets installed on the truck bed or the like, there may be cases where the packages or pallets cannot be accurately detected. When loading or unloading packages installed on the truck bed or the like, a technology that can quickly and accurately grasp the number of packages or pallets in a simpler way compared to packages installed in the warehouse is desired.

[0006] The present invention has been made to solve the above problems, and an object thereof is to provide a pallet detection device that can quickly and accurately grasp the number of pallets installed on the truck bed in a simpler way.

Means for Solving the Problems

[0007] In order to solve the problems as described above, the pallet detection device of the present invention includes an imaging unit that acquires first image information and second image information with different imaging angles by imaging images of pallets loaded on the truck bed from different angles, and compares the first image information and the second image information with the image information of the pallets registered in advance, and detects the image information of the pallets from each of the first image information and the second image information to obtain pallet information including the type information, position information, size information, and reliability coefficient of pallet detection of the detected pallets, and an image processing unit that obtains third image information and fourth image information having a predetermined shape by performing projective transformation on each of the first image information and the second image information, and obtains the position information and the size information of the pallets converted along with the projective transformation, and a pallet determination unit that determines the pallet information with a high reliability coefficient from the pallet information of the pallets loaded at the same position on the truck bed in the third image information and the fourth image information.

[0008] Further, the pallet determination unit may determine whether the pallet is loaded at the same position on the loading platform based on the difference between the position information and the size information of the pallet in the third image information and the position information and the size information of the pallet in the fourth image information.

Advantages of the Invention

[0009] According to the present invention, it is possible to provide a pallet detection device capable of quickly and accurately grasping the number of pallets installed on the loading platform by a simpler method.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the present invention can be implemented in various embodiments and is not limited to the embodiments of the invention described below.

[0012] <Configuration of Pallet Detection System> FIG. 1 is an example of the configuration of a pallet detection system according to an embodiment of the present invention. The pallet detection system in FIG. 1 includes a pallet detection device 10 that acquires an image of a pallet 3 with a load 2 placed thereon on the loading platform of a transport vehicle such as a truck 1, and detects the pallet 3 on the loading platform using the acquired image.

[0013] The pallet detection system in FIG. 1 is configured to detect a pallet 3 with a load 2 placed thereon on the loading platform of the truck 1 by comparing the image information acquired by the imaging unit mounted on the pallet detection device 10 with the pre-registered image information of the pallet.

[0014] When loading or unloading the load 2 installed on the loading platform of the truck 1 or the like, it is necessary to detect the number of pallets 3 in the state where the load is installed on the loading platform of the truck 1 or the like to determine whether the load to be transported is loaded. Therefore, compared with the case of checking the load installed in the warehouse, there are time or location constraints, so there is a demand to accurately grasp the number of loads and pallets quickly and accurately by a simpler method.

[0015] In the configuration example of FIG. 1, on the loading platform of the truck 1, a plurality of pallets 3 with a plurality of loads 2 placed thereon are arranged in a stacked state. Since the types of pallets 3 on the loading platform of the truck 1 are not limited to one type, it is necessary to grasp the types and numbers of a plurality of types of pallets by acquiring an image of the entire load on the loading platform of the truck 1.

[0016] The pallet detection device 10 acquires image information of the entire loading platform including an image of the pallet 3, and detects an image of the pallet from the image information of the entire loading platform by comparing it with the pre-stored image information of the pallet. The pre-stored image information of the pallet includes information representing the characteristics of the pallet, such as the color and shape of the pallet.

[0017] As the image information of the palette to be saved in advance, images taken for each type of palette may be saved, or image information learned using the image information of a large number of palettes may be used. By detecting information representing the characteristics of the palette, such as the color and shape of the palette, from the image information of the entire loading platform including the image information of Palette 3, it is possible to grasp the type and number of palettes on the loading platform of the truck 1.

[0018] Here, when automatically detecting the type and number of palettes loaded on the loading platform of the truck from the captured image by the camera, if it is possible to obtain a captured image from the front of the truck loading platform, the palette can be detected relatively easily by using the image information obtained by performing machine learning using the pre-prepared learning image. However, in the truck bay, since the shooting space is limited, it is often impossible to secure a sufficient shooting distance for the truck loading platform. In that case, in order to obtain an image of the entire truck loading platform, the truck loading platform is photographed obliquely, and the detection accuracy of the palette far from the shooting position may decrease.

[0019] The palette detection device 10 of the present embodiment is configured to perform shooting from two positions with different angles with respect to the loading platform of the truck, perform projective transformation on the image information taken from two positions with different angles so as to become a front image, and at the palette positions detected in the two images subjected to projective transformation, adopt palette information with high reliability of palette detection as the palette detection result.

[0020] By performing palette detection using two pieces of image information taken from two positions with different angles with respect to the truck loading platform, even when the shooting space is limited as in the truck bay, it is possible to accurately grasp the type and number of palettes using the camera captured image.

[0021] <Configuration of Palette Detection Device> The configuration of the palette detection device 10 will be described with reference to FIG. 2. FIG. 2 is an example of a functional block of the palette detection device in the embodiment of the present invention.

[0022] The pallet detection device 10 includes an imaging unit 11 for acquiring image information of a loading platform including image information of a pallet, a central processing unit 12 for performing image processing, pallet detection, and pallet determination on the image acquired by the imaging unit 11, a storage unit 16 for storing various information, etc., an input unit 17 for an operator to input information, and a display unit 18 for displaying the detection status of the image of the pallet.

[0023] The imaging unit 11 is an image sensor for acquiring image information of the entire loading platform including an image of the pallet 3. As the imaging unit 11, a general-purpose RGB camera for detecting the color and shape of the pallet from the image information of the entire loading platform can be used. As the pallet detection device 10 provided with the imaging unit, for example, a digital camera equipped with an RGB camera, a smartphone, or the like can be used.

[0024] The central processing unit 12 functions as a pallet detection unit 13 for detecting a pallet from the image acquired by the imaging unit 11, an image processing unit 14 for performing image processing such as projective transformation of the acquired image, and a pallet determination unit 15 for determining highly reliable pallet information from the detected pallet information. The pallet detection device 10 can be configured by a computer including a CPU, a memory, and an interface for connecting to external devices.

[0025] The storage unit 16 can store pallet information such as image information of the pallet for comparison with the acquired image information of the loading platform, type information of the pallet, position information, size information, and a reliability coefficient for pallet detection. The image information of the pallet includes information for representing characteristics of the pallet such as the color and shape of the pallet.

[0026] <Detection of Pallet Information> FIG. 3 is a diagram for explaining a method for detecting pallet information in an embodiment of the present invention. In the example of FIG. 3, image information (A, B, C, D) of the loading platform is acquired from different left and right angles with respect to the loading platform, pallet detection is performed on the acquired image information, and projective transformation of the image information is performed.

[0027] In FIG. 3, image information (A, B, C, D) of the loading platform is acquired from different left and right angles with respect to the loading platform, and by comparing the left image information (first image information) and the right image information (second image information) with the image information of a pre-registered pallet, the image information of the pallet is detected from each of the left image information and the right image information, and pallet information including the type information, position information, size information, and reliability coefficient of pallet detection of the detected pallet is obtained.

[0028] As illustrated in FIG. 3, it is desirable to acquire the image information of the loading platform from the left and right angles from the front direction of the loading platform. In particular, when the distance from the photographable loading platform is short, it is assumed that the reliability of pallet detection decreases as the distance from the photographing position increases. Therefore, by acquiring the image of the entire loading platform from each of the left and right photographing positions from the front direction of the loading platform and performing pallet detection using the two pieces of image information obtained at the left and right photographing positions, it becomes possible to obtain more reliable pallet information.

[0029] The type information of the pallet is specified based on the characteristics of the pallet such as the color and shape of the pre-registered pallet. The position information of the pallet is the relative pixel position information of the pallet in the acquired image information, and can be represented by XY coordinates with A in the image information (A, B, C, D) of the loading platform in FIG. 3 as the origin. Similarly, the size information of the pallet can be represented as the relative pixel size information in the acquired image information.

[0030] The reliability coefficient of pallet detection is a quantification of the degree of approximation in "color", "shape", "image contrast", etc. when comparing the acquired image information with the image information of the pre-registered pallet. For example, the case of [reliability coefficient = 1] has the highest reliability, and the case of [reliability coefficient = 0] has the lowest reliability.

[0031] When the shooting space of the loading platform is limited like a track bar, the loading platform is photographed from an oblique direction instead of the front position with respect to the loading platform. Therefore, as illustrated in FIG. 3, the image information is acquired in a distorted state. When a pallet image is detected in such image information, it is assumed that the reliability of pallet detection decreases as the distance from the shooting position increases.

[0032] In the present embodiment, as shown in FIG. 3, shooting is performed from two positions in different directions with respect to the loading platform at different angles, and the left image information (first image information) and the right image information (second image information) obtained by shooting are subjected to projective transformation so as to become a front image, and image information (third image information and fourth image information) having a predetermined shape is acquired.

[0033] Along with this projective transformation, the position information and size information of the pallet detected in the left image information (first image information) and the right image information (second image information) are transformed along with the projective transformation of the image information.

[0034] In the present embodiment, at the pallet position in the image information (third image information and fourth image information) corresponding to the left image information and the right image information, which is obtained by performing projective transformation, pallet information with high reliability of pallet detection is determined, and the type and number of pallets loaded on the loading platform are grasped based on the pallet information with high reliability.

[0035] <Determination of pallet information> FIG. 4 is a diagram for explaining the detection result of pallet information in the embodiment of the present invention. FIG. 4 shows the pallet information at the pallet position in the two pieces of image information subjected to projective transformation in FIG. 3.

[0036] In FIG. 4, x and y represent the coordinate positions of the pallet, w and h represent the size (width and height) of the pallet, Δ(x, y) represents the Euclidean distance between the coordinate positions in the left and right image information, and Δ(w, h) represents the area ratio of the pallet images in the left and right image information. The coordinate position (x, y) and the pallet size (w, h) of the pallet are those converted along with the projective transformation of the image information.

[0037] In this embodiment, it is possible to determine that the pallets are at the same pallet position based on the differences in the coordinate positions and the pallet sizes of the pallets in the left and right image information. For example, it may be configured to determine that the pallets are at the same pallet position when Δ(x, y) and Δ(w, h) are equal to or less than a predetermined threshold.

[0038] In this embodiment, the pallet reliability coefficients of the pallet information at the same pallet position in FIG. 4 are compared, the pallet information with a high reliability coefficient is determined, and the type and number of pallets loaded on the loading platform are grasped based on the pallet information with a high reliability coefficient. For example, at the pallet position [X0, Y0], the pallet information of the left image with a high reliability coefficient is adopted. The same applies to other pallet positions. In the example of FIG. 4, it is detected that there are "2" pallets of the pallet type "green", "6" pallets of the pallet type "orange", and "2" pallets of the pallet type "blue".

[0039] <Operation flow of pallet detection device> The pallet detection operation in the pallet detection device 10 will be described with reference to FIG. 5. FIG. 5 is an example of the operation flow of the pallet detection method in the embodiment of the present invention.

[0040] The imaging unit 11 of the pallet detection device 10 acquires left image information and right image information with different imaging angles by imaging the image of the pallet loaded on the loading platform from different angles (S1-1).

[0041] The pallet detection unit 13 of the pallet detection device 10 detects the pallet image information from each of the left image information and the right image information by comparing the left image information and the right image information with the pre-registered pallet image information, and obtains pallet information including the type information, position information, size information of the detected pallet, and the reliability coefficient of the pallet detection (S1-2).

[0042] The image processing unit 14 of the pallet detection device 10 obtains image information having a predetermined shape by performing projective transformation on each of the left image information and the right image information, and obtains the position information and size information of the pallet converted along with the projective transformation (S1-3).

[0043] The pallet determination unit 15 of the pallet detection device 10 performs alignment of the projective-transformed image information, and determines the pallet information with a high reliability coefficient from the pallet information loaded at the same pallet position in the projective-transformed image information (S1-4, S1-5).

[0044] According to the present embodiment, pallet detection is performed using two pieces of image information taken from two directions with different angles with respect to the truck bed, the pallet information with a high reliability coefficient is determined in the image information obtained by performing projective transformation on the two pieces of image information, and the type and number of pallets loaded on the truck bed are grasped based on the pallet information with a high reliability coefficient. Therefore, even when the imaging space is limited as in the case of a truck berth, it is possible to accurately detect the type and number of pallets loaded on the truck bed using the camera-captured image.

Description of Reference Numerals

[0045] 1... truck, 2... luggage, 3... pallet, 10... pallet detection device, 11... imaging unit, 12... central processing unit, 13... pallet detection unit, 14... image processing unit, 15... pallet determination unit, 16... storage unit, 17... input unit, 18... display unit.

Claims

1. an imaging unit that captures images of a pallet loaded on a loading platform from different angles to obtain first image information and second image information captured at different imaging angles; a palette detection unit that detects image information of the palette from the first image information and the second image information by comparing the first image information and the second image information with image information of the palette registered in advance, and acquires palette information including type information, position information, size information, and a palette detection reliability coefficient of the detected palette; an image processing unit that performs projective transformation on the first image information and the second image information to obtain third image information and fourth image information having predetermined shapes, and obtains the position information and the size information of the palette transformed in association with the projective transformation; a pallet determination unit that determines, from the pallet information of the pallets loaded at the same position on the loading platform in the third image information and the fourth image information, the pallet information having a high reliability coefficient; A pallet detection device comprising:

2. The pallet determination unit determines whether the pallet is loaded at the same position on the loading platform based on a difference between the position information and the size information of the pallet in the third image information and the position information and the size information of the pallet in the fourth image information.

2. The pallet detection device according to claim 1.

Images