Pallet detection device and pallet detection method

The pallet detection device and method address the challenge of accurately counting pallets on loading platforms by employing image processing and geometric transformations to correct distortions, ensuring precise and efficient pallet detection.

JP2024158710AActive Publication Date: 2024-11-08UPR CORP

Patent Information

Application Number
JP2023074123
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08
Estimated Expiration
2043-04-28

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately and quickly determine the number of pallets on a loading platform, especially under varying imaging conditions, leading to potential inaccuracies and duplicate counting.

Method used

A pallet detection device and method utilizing an imaging unit, image processing, and geometric transformation to correct image distortions, enabling accurate detection of pallets by comparing acquired images with pre-registered pallet information, and calculating pallet dimensions using vertex coordinates.

Benefits of technology

Enables quick and accurate determination of pallet counts on loading platforms, improving detection accuracy and eliminating duplicate counts by correcting image distortions and using geometric transformations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158710000001_ABST
    Figure 2024158710000001_ABST
Patent Text Reader

Abstract

To provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler manner.SOLUTION: A pallet detection device 10 of the present invention includes: an imaging unit 11 that acquires first image information including an image of a pallet placed on a loading platform; an image processing unit 13 that acquires second image information having a predetermined shape by geometrically transforming a shape of the first image information; and a pallet detection unit 14 that detects a pallet image from the second image information by comparing the second image information with image information of a pallet registered in advance. The pallet detection unit 14 determines, on the basis of at least one of information on a pallet width and a pallet height, a relative position of the pallet on the loading platform.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a pallet detection technique for detecting a pallet placed on a loading platform. [Background technology]

[0002] As a technology for managing luggage in a warehouse, a method has been proposed to manage the locations of luggage and pallets in the warehouse by reading the identification codes attached to the luggage and pallets.Recently, a technology has been proposed to reduce the burden of inspecting and inventorying luggage in the warehouse by acquiring image information of luggage and pallets in the warehouse using cameras mounted on transport vehicles or drones that move around the warehouse.

[0003] For example, Patent Documents 1 and 2 propose a pallet detection system that uses a robot or drone equipped with a camera that takes images of luggage in a warehouse. By reading barcodes on luggage or shelves with a camera mounted on the robot or drone, information on the type and number of luggage can be obtained. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-527172 A [Patent Document 2] JP 2019-167177 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, in the above-mentioned technology for reading barcodes attached to luggage or pallets, it is necessary to acquire images of the luggage or pallet from a relatively short distance and from the front direction, so when inspecting luggage or pallets placed on the bed of a truck, etc., the luggage or pallet may not be detected accurately. When luggage placed on the bed of a truck, etc. is brought in or taken out, there is a demand for a technology that can accurately determine the number of luggage or pallets in a simpler manner, more quickly and accurately, compared to luggage placed in a warehouse.

[0006] The present invention has been made to solve the above-mentioned problems, and has an object to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler manner. [Means for solving the problem]

[0007] In order to solve the problems described above, the pallet detection device of the present invention comprises an imaging unit that acquires first image information including an image of a pallet loaded on a loading platform, an image processing unit that acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information, and a pallet detection unit that detects the image of the pallet from the second image information by comparing the second image information with image information of the pallet that has been registered in advance, and the pallet detection unit determines the relative position of the pallet on the loading platform based on information on at least one of the width and height of the pallet.

[0008] The palette detection unit may also calculate information on at least one of a width of the palette and a height of the palette, using vertex coordinates of constituent surfaces of the palette.

[0009] In order to solve the problems described above, the pallet detection method of the present invention is a pallet detection method executed in a pallet detection device including an imaging unit that acquires an image of the pallet, an image processing unit that performs image processing on the image acquired by the imaging unit, and a pallet detection unit that detects an image of the pallet from the image processed by the image processing unit, and includes an image acquisition step in which the imaging unit acquires first image information including an image of the pallet loaded on a loading platform, an image processing step in which the image processing unit acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information, and a pallet image detection step in which the pallet detection unit detects the image of the pallet from the second image information by comparing the second image information with image information of the pallet that has been registered in advance, and the pallet image detection step determines the relative position of the pallet on the loading platform based on information on at least one of the width and height of the pallet.

[0010] The palette image detection step may include calculating information on at least one of a width and a height of the palette using vertex coordinates of constituent surfaces of the palette. Effect of the Invention

[0011] According to the present invention, it is possible to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler manner. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 shows an example of a system configuration including a pallet detection device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining an example of palette detection using the palette detection device according to the embodiment of the present invention. [Diagram 3] FIG. 3 is an example of a functional block diagram of the pallet detection device according to the embodiment of the present invention. [Figure 4]FIG. 4 is a diagram for explaining the geometric transformation of the loading platform image according to the embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram for explaining detection of a palette according to the embodiment of the present invention. [Figure 6] FIG. 6 shows an example of an operational flow of the palette detection method according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the present invention can be embodied in various forms and is not limited to the embodiment of the invention described below.

[0014] <Pallet detection system configuration> Fig. 1 shows an example of a system configuration including a pallet detection device according to an embodiment of the present invention. The pallet detection system in Fig. 1 is composed of a pallet detection device 10 that acquires an image of a pallet 3 on which luggage 2 is placed, which is placed on the loading platform of a transport vehicle such as a truck 1, and detects the number of pallets 3 on the loading platform using the acquired image, and a pallet management server 20 that manages the number of pallets detected by the pallet detection device 10.

[0015] The pallet detection system in Fig. 1 is configured to detect a pallet 3 on which luggage 2 is placed that is loaded on the bed of a truck 1 by comparing image information acquired by an imaging unit mounted on a pallet detection device 10 with image information of a pallet registered in advance. The image of the detected pallet 3 and the detection result of the pallet 3 are transmitted to a pallet management server 20 as necessary.

[0016] In order to determine whether or not luggage 2 placed on the bed of a truck 1 or the like is loaded when the luggage 2 is loaded or unloaded, it is necessary to detect the number of pallets 3 when the luggage 2 is loaded on the bed of a truck 1 or the like. Therefore, compared to checking luggage placed in a warehouse, there are time and space constraints, and therefore a technology is required to quickly and accurately determine the number of pallets 3 in a simpler manner. The pallet detection device 10 of this embodiment is configured to determine the number of pallets 3 on which multiple luggage 2 are placed, using an image of the entire luggage including an image of the pallet 3.

[0017] The pallet detection device 10 acquires image information of the entire loading platform, including an image of the pallet 3, and compares it with pre-stored image information of the pallet to detect the image of the pallet from the image information of the entire loading platform. The pre-stored image information of the pallet includes information representing the characteristics of the pallet, such as the shape of the pallet.

[0018] As the image information of the pallets 3, images taken for each type of pallet 3 may be stored, or image information learned using image information of a large number of pallets may be used. The types and number of pallets 3 on the loading platform may be ascertained by detecting information representing the characteristics of the pallets, such as the color and shape of the pallets 3, from image information of the entire loading platform including the image information of the pallets 3.

[0019] <Geometric transformation of the loading platform image> When loading or unloading luggage placed on the bed of a truck 1 or the like, the photographer taking the image of the bed is not necessarily in the same position, so the distance and angle between the digital camera or smartphone taking the image of the bed and the bed are not constant depending on the photographer.

[0020] Therefore, in the acquired loading platform image, the image information of the pallet stored in advance cannot be detected, and the detection accuracy of the pallet image may decrease. The pallet detection device 10 of the present embodiment is configured to improve the detection accuracy of the pallet image by performing image processing such as geometric transformation on the acquired loading platform image.

[0021] Specifically, if the distance and angle between the digital camera or smartphone and the loading platform differ depending on the photographer taking the image of the loading platform, distortion will occur in the shape of the loading platform image. Therefore, by geometrically transforming the shape of the acquired loading platform image (first image information), the image distortion is corrected so that the captured image has a specified shape, and the image information of the pallet is detected using the image (second image information) having the specified shape with the distortion corrected.

[0022] Here, if the image having the specified shape with the distortion corrected does not correspond to the specified angle of view, the imaging unit may be zoomed in or out as necessary to acquire information about the loading platform image again, and image processing such as geometric transformation may be performed on the acquired loading platform image to acquire an image having the specified shape with the distortion corrected.

[0023] In this way, by correcting image distortion so that the captured image has a specified shape, and comparing the image information of the pallet with a loading platform image corresponding to a specified angle of view, the detection accuracy of the image information of the pallet can be improved when detecting the image information of the pallet from the corrected loading platform image.

[0024] As a method of geometric transformation of the loading platform image, for example, keystone correction for correcting a trapezoidal image to a rectangular image is conceivable, but it is not limited to keystone correction. Image transformation may be performed so that a rectangular image of any shape is converted to an image corresponding to a predetermined angle of view, without being limited to keystone correction.

[0025] When an image of the required size is obtained by the geometric transformation, it is not necessarily necessary to zoom in or out the imaging unit. As an image corresponding to a predetermined angle of view for detecting image information of a pallet, it is possible to zoom in or out the imaging unit so that the area of ​​the image is approximately the same as the image size at the predetermined angle of view, or to zoom in or out the imaging unit so that the image size is within a predetermined range at the predetermined angle of view. The imaging unit may be zoomed in or out so that the image size corresponds to the image information of the loading platform that has been learned in advance.

[0026] <Detecting the position of pallets on a loading platform> Fig. 2 is a diagram for explaining an example of pallet detection using a pallet detection device according to an embodiment of the present invention. In the example of Fig. 2, the loading platform of truck 1 has a structure divided into two sections, left and right, and each of the left and right sections has a depth equivalent to one pallet.

[0027] In the case of a truck 1 bed structure divided into two sections, left and right, as shown in Figure 2, it is possible to determine the number of pallets in the entire bed by detecting pallets using images acquired from the outside of each of the left and right beds.

[0028] Here, when detecting pallets from pallet detection device #1 on the left side of truck 1, if a loading platform image is acquired while a pallet (pallet P in FIG. 2) placed in the right section of the loading platform is visible, pallet detection device #2 on the right side of the loading platform of truck 1 may erroneously detect pallet P that should be detected. Pallet detection devices #1 and #2 may count pallets P in duplicate, making it impossible to accurately determine the number of pallets on the loading platform. Thus, the pallet detection device of this embodiment is configured to determine the position of the pallet on the loading platform when pallets 3 are placed on a loading platform with depth as shown in FIG. 2, thereby making it possible to accurately determine the number of pallets.

[0029] <Configuration of Pallet Detector> The configuration of the pallet detection device 10 will be described with reference to Fig. 3. Fig. 3 shows an example of a functional block diagram of the pallet detection device according to the embodiment of the present invention.

[0030] The pallet detection device 10 comprises an imaging unit 11 for acquiring image information of the loading platform including image information of the pallet, a central processing unit 12 for performing image processing of the image acquired by the imaging unit 11 and for pallet detection, a wireless unit 18 for sending and receiving information to the pallet management server 20, a memory unit 15 for storing various information, etc., an input unit 16 for the photographer to input information, and a display unit 17 for displaying the detection status of the pallet image.

[0031] The imaging unit of the pallet detection device 10 is an imaging element for acquiring image information of the entire loading platform, including an image of the pallet. A general-purpose RGB camera for detecting the shape and size of the pallet from the image information of the entire loading platform can be used as the imaging unit. For example, a digital camera or a smartphone equipped with an RGB camera can be used as the pallet detection device 10 equipped with an imaging unit.

[0032] The central processing unit 12 functions as an image processing unit 13 that performs image processing of the image captured by the imaging unit 11, and as a palette detection unit 14 that detects palettes. The palette detection device 10 can be configured by a computer equipped with a CPU, a memory, and an interface for connecting to external devices.

[0033] The storage unit 15 can store image information of the pallet and the detection result of the pallet for comparison with the acquired image information of the loading platform. The image information of the pallet includes information for expressing the characteristics of the pallet such as the color and shape of the pallet. The image information of the pallet may use image data obtained by learning image data of a plurality of pallets.

[0034] <How to detect a pallet> A pallet detection method in an embodiment of the present invention will be described with reference to Figures 4 and 5. Figure 4 is a diagram for explaining the geometric transformation of a loading platform image in an embodiment of the present invention, and Figure 5 is a diagram for explaining the detection of a pallet in an embodiment of the present invention.

[0035] In the example of Figure 4, a geometric transformation is performed so that the reference position (a, b, c, d) on the platform becomes a rectangular image (A, B, C, D). As a result of the geometric transformation, an image is obtained in which the area of ​​the image for detecting the luggage image is within a specified range at a specified angle of view.

[0036] In Fig. 4, the loading platform image obtained by geometric transformation in imaging unit 11 is within a predetermined range at a predetermined angle of view, and is an image with a shape and size that can be compared with the image information of pallets 3. By using this image and comparing it with the image information of pallets 3, the type and number of pallets 3 can be identified.

[0037] Here, if the image acquired by the imaging unit 11 and subjected to geometric transformation does not have a shape and size that can be compared with the image information of the palette 3, the imaging unit 11 may perform the geometric transformation again using an image acquired by zooming in or out as necessary.

[0038] 5 is an example of the result of detecting the number of pallets using the relative positions of the pallets. The pallet detection device of this embodiment is configured to detect the pallets 3 on the loading platform by using an image obtained by performing geometric transformation on the image information acquired by the imaging unit 11 and comparing the image information of the pallets 3 stored in advance, and further to calculate the width and height of the pallet 3 using the vertex coordinates of the constituent faces of the detected pallet 3, and to detect the number of pallets 3 on the front side by grasping the relative position of the pallet 3 using the calculated width and height information of the pallet 3.

[0039] In this embodiment, the pallets 3 are classified according to the width and height of the pallets 3, and the relative positions of the pallets 3 are grasped, thereby eliminating duplicate detection of the pallets 3. For example, the width and height of the pallets 3 are compared with a predetermined threshold, and the pallets 3 are classified according to whether the width and height of the pallets 3 are larger or smaller than the predetermined threshold, and the pallets having widths and heights larger than the predetermined threshold are determined as the pallets in the front, and the other pallets are determined as the pallets in the back. In the example of FIG. 5, the number of pallets 3 in the front, which have a relatively large width and height, is "8", and the number of pallets 3 in the back, which have a relatively small width and height, is "3".

[0040] It is not necessary to use both the width and height information of the pallet 3 in order to grasp the relative position of the pallet 3. The relative position of the pallet 3 can be grasped using at least one of the width and height information of the pallet 3.

[0041] According to this embodiment, even if the distance or angle between the digital camera or smartphone and the loading platform is not constant depending on the photographer who takes the image of the loading platform of the truck 1 or the like, it is possible to improve the detection accuracy of pallet images of luggage placed on the loading platform of the truck 1 or the like by performing image processing such as geometric transformation on the acquired image. Furthermore, by determining the relative positions of the pallets 3 on the loading platform based on information on the width and height of the pallets 3 on the loading platform, it is possible to eliminate detection of duplicated pallets 3 and accurately grasp the number of pallets 3 on the loading platform.

[0042] <Operation flow of the pallet detection device> The operation of the palette detection method executed in palette detection device 10 will be described with reference to Fig. 6. Fig. 6 shows an example of the operation flow of the palette detection method in the embodiment of the present invention.

[0043] The imaging unit of the pallet detection device acquires an image of the loading platform including an image of the pallet (S1-1: image acquisition step). The image processing unit of the pallet detection device detects a loading platform area including the image of the pallet from the acquired image (S1-2) and performs a geometric transformation of the loading platform image so that the shape of the image of the loading platform area becomes a predetermined shape (S1-3: image processing step).

[0044] The pallet detection unit of the pallet detection device compares the image of the loading platform area after geometric transformation with image information of the pallet registered in advance (S1-4), detects the image of the pallet from the loading platform image, and detects the vertex coordinates of the constituent surfaces of the pallet (S1-5: pallet image detection step).

[0045] The pallet detection unit of the pallet detection device further calculates the width and height of the pallet using the vertex coordinates of the constituent surfaces of the pallet (S1-6), and determines the relative position of the pallet using at least one of the pallet width and height information, and calculates the number of pallets in the front side (S1-7).

[0046] The image in which the palette is detected and the detection result of the palette are transmitted to the palette management server as necessary. The detection result is verified by verifying the image in which the palette is detected and the detection result of the palette, and if the detection result is incorrect, the palette image may be trained using the corrected detection result to improve the accuracy of detection.

[0047] For example, if a user refers to an image in which a palette has been detected and the palette detection results, and it is determined that an image that is not a palette has been detected as a palette (false detection), or conversely, an image that should have been detected as a palette has not been detected (missed detection), the accuracy of detection can be improved by learning the falsely detected images as images other than palettes and the missed detection images as images of palettes.

[0048] In this way, according to this embodiment, it is possible to provide a pallet detection device that can quickly and accurately determine the number of pallets placed on a loading platform in a simpler manner. [Explanation of symbols]

[0049] 1...truck, 2...baggage, 3...pallet, 10...pallet detection device, 11...imaging unit, 12...central processing unit, 13...image processing unit, 14...pallet detection unit, 15...memory unit, 16...input unit, 17...display unit, 20...pallet management server.

Claims

1. an imaging unit that acquires first image information including an image of a pallet loaded on a loading platform; an image processing unit that acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information; a palette detection unit that detects an image of the palette from the second image information by comparing the second image information with image information of the palette registered in advance; Equipped with The pallet detection unit determines the relative position of the pallet on the loading platform based on information on at least one of the width and height of the pallet. Pallet detection device.

2. The pallet detection unit calculates information on at least one of the width and height of the pallet using vertex coordinates of the constituent surfaces of the pallet.

2. The pallet detection device according to claim 1.

3. A palette detection method executed in a palette detection device including an imaging unit that acquires an image of a palette, an image processing unit that processes the image acquired by the imaging unit, and a palette detection unit that detects the image of the palette from the image processed by the image processing unit, an image acquisition step in which the imaging unit acquires first image information including an image of the pallet loaded on the loading platform; an image processing step in which the image processing unit geometrically transforms the shape of the first image information to obtain second image information having a predetermined shape; a palette image detection step in which the palette detection unit compares the second image information with image information of the palette registered in advance to detect an image of the palette from the second image information; and The pallet image detection step determines the relative position of the pallet on the loading platform based on information on at least one of the width and height of the pallet. Palette detection method.

4. The palette image detection step calculates information on at least one of the palette width and the palette height using vertex coordinates of the palette's constituent surfaces. The palette detection method according to claim 3 .

Citation Information

Patent Citations

  • Autonomous traveling vehicle, and object recognition method in autonomous traveling vehicle

    JP2015225450A

  • Position and attitude estimation device

    JP2021024718A

  • Loading and unloading system

    JP2021088440A

  • Freight handling assistance system

    JP2021160860A

  • Pallet detection device and pallet detection method

    JP7185958B1

Cited By

  • Pallet detection system

    JP2025173599A

  • Pallet Detection System

    JP7658630B1

  • Package information management system and package information management method

    JP7870103B1