Pallet detection system

Abstract

To provide a pallet detection system capable of quickly and accurately detecting pallets loaded on a loading platform in a simpler manner.SOLUTION: A pallet detection system includes a pallet detection device 10 that detects images of pallets 3 from image information including images of the pallets 3 loaded on the loading platform of a transport vehicle 1, acquires pallet information including position information, type information, and number information of the pallets 3 on the loading platform, and corrects the acquired pallet information, and a pallet management device 20 that manages the corrected pallet information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a pallet detection system that detects a pallet placed on a loading platform. [Background technology]

[0002] As a technology for managing cargo in warehouses, a method has been proposed in which the locations of cargo and pallets within the warehouse are managed by reading the identification codes attached to the cargo and pallets.Recently, a technology has been proposed that reduces the workload of inspecting cargo and taking inventory within the warehouse by acquiring image information of the cargo and pallets within the warehouse using cameras mounted on guided vehicles or drones that move within the warehouse.

[0003] For example, Patent Documents 1 and 2 propose a pallet detection system that uses a robot or drone equipped with a camera that takes images of packages in a warehouse. By reading barcodes attached to packages or shelves with a camera mounted on the robot or drone, information on the type and number of packages can be obtained. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-527172 [Patent Document 2] Japanese Patent Application Publication No. 2019-167177 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, when detecting pallets when luggage placed on the loading platform of a transport vehicle or the like is being brought into or taken out of a warehouse, there are time and location constraints compared to detecting luggage or pallets placed in a warehouse, so it is necessary to quickly and accurately grasp the luggage or pallet.

[0006] Furthermore, the technology for detecting luggage and pallets as described above requires equipment to detect pallets, such as robots or drones equipped with cameras that take images of luggage and pallets, which creates the problem of placing a heavy burden on workers when detecting luggage and pallets.

[0007] The present invention has been made to solve the above problems, and aims to provide a pallet detection system that can quickly and accurately identify pallets loaded on the loading platform of a transport vehicle in a simpler way. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the pallet detection system of the present invention is a pallet detection system comprising a pallet detection device that detects pallets on a loading platform, and a pallet management device that manages information about the pallets using the detection results of the pallet detection device, wherein the pallet detection device comprises an imaging unit that acquires first image information including an image of a pallet loaded on the loading platform, a pallet detection unit that detects an image of the pallet from the first image information by comparing the first image information with second image information of the pallet that has been registered in advance, a pallet information acquisition unit that acquires first pallet information including image information of the pallet, position information of the pallet on the loading platform, type information of the pallet, and information about the number of pallets based on the detection results of the pallet detection unit, a pallet information correction unit that corrects the first pallet information, and a transmission unit that transmits second pallet information obtained by correcting the first pallet information to the pallet management device.

[0009] In addition, the pallet information correction unit may detect falsely detected pallets and / or missed-detection pallets in the first pallet information based on specific position information of the pallet on the loading platform that has been registered in advance, and correct the first pallet information to the second pallet information.

[0010] In addition, the palette detection device may include a display unit that displays the first palette information and an input unit that inputs correction information for the first palette information, and the palette information correction unit may correct the first palette information to the second palette information based on the correction information in the input unit.

[0011] In addition, the palette management device may generate the second image information of the palette by learning image information of the palette using the second palette information, and provide the generated second image information to the palette detection device.

[0012] The transmitting unit of the pallet detection device may transmit the first pallet information of the falsely detected pallet and / or the missed detection pallet to the pallet management device in the pallet information correction unit, and the pallet management device may generate the second image information of the pallet by learning the second pallet information and the first pallet information of the falsely detected pallet and / or the missed detection pallet, and provide the generated second image information to the pallet detection device. [Effects of the Invention]

[0013] According to the present invention, it is possible to provide a pallet detection system that can quickly and accurately detect pallets loaded on the loading platform of a transport vehicle using a simpler method. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 shows an example of the system configuration of a pallet detection system according to an embodiment of the present invention. [Figure 2] FIG. 2 is an example of a functional block diagram of a pallet detection device according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram illustrating the detected palette information according to the embodiment of the present invention. [Figure 4]FIG. 4 is a diagram illustrating the palette information after correction according to the embodiment of the present invention. [Figure 5] FIG. 5 shows an example of an operation sequence of the pallet detection system according to the embodiment of the present invention. [Figure 6] FIG. 6 shows an example of an operation flow of the pallet detection device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention can be embodied in various embodiments and is not limited to the embodiments of the invention described below.

[0016] <Configuration of Pallet Detection System> Figure 1 shows an example of the configuration of a pallet detection system according to an embodiment of the present invention. The pallet detection system in Figure 1 is composed of a pallet detection device 10 that acquires an image of a pallet 3 on which cargo 2 is placed, which is placed on the loading platform of a transport vehicle 1, and detects the pallet 3 on the loading platform using the acquired image, and a pallet management device 20 that manages pallet information including the image, position, type, and number of pallets detected by the pallet detection device 10.

[0017] The pallet detection system in Fig. 1 is configured to detect pallets 3 on which cargo 2 is placed that are loaded on the loading platform of a transport vehicle 1 by comparing image information acquired by an imaging unit mounted on a pallet detection device 10 with image information of pre-registered pallets. Images of the detected pallets 3 and pallet information including the position, type, and number of detected pallets 3 are sent to a pallet management device 20. The pallet information is used for pallet image learning in the pallet management device 20.

[0018] When detecting the luggage 2 or pallets 3 when luggage 2 placed on the loading platform of a transport vehicle 1 or the like is being carried in or out, it is necessary to check the luggage 2 or pallets 3 being carried in or out while they are placed on the loading platform of the transport vehicle 1 or the like. Therefore, compared to checking the luggage 2 or pallets 3 while they are placed in a warehouse, there are time and location constraints, and there is a demand for a simpler method to quickly and accurately determine the number of luggage or pallets.

[0019] In the system configuration example of Fig. 1, as shown in the top view of Fig. 1, pallets 3 on which a plurality of packages 2 are placed are arranged in multiple layers on the loading platform of the transport vehicle 1. Since the type of pallets 3 on the loading platform of the transport vehicle 1 is not limited to one type, it is necessary to acquire an image of the entire loading platform of the transport vehicle 1 to ascertain the types and number of the multiple types of pallets 3.

[0020] The loading platform of the transport vehicle 1 in the system configuration example of Figure 1 has a structure divided into two sections, left and right, as shown in the rear view of Figure 1, and each of the left and right sections has a depth equivalent to one pallet. In a loading platform configured in this way, the number of pallets 3 on the entire loading platform can be determined by detecting the pallets using images acquired from the outside of the loading platform on each of the left and right platforms.

[0021] The imaging unit of the pallet detection device 10 is an imaging element for acquiring image information of the pallet 3. Since the imaging unit of the pallet detection device 10 needs to acquire image information of the entire loading platform including the pallet 3, it acquires an image of the loading platform so that an image of the reference position (ABCD in FIG. 1) of the loading platform image is included.

[0022] A general-purpose RGB camera for detecting the color and shape of a pallet from image information of the entire loading platform can be used as the imaging unit of the pallet detection device 10. For example, a digital camera or smartphone equipped with an RGB camera can be used as the pallet detection device 10 equipped with an imaging unit.

[0023] The pallet detection device 10 acquires image information of the entire loading platform including an image of the pallet 3, and compares the acquired image information with image information of the pallet 3 stored in advance, thereby detecting the image of the pallet 3 from the image information of the entire loading platform.

[0024] The image information of the palette 3 to be compared includes information representing the palette's characteristics such as the palette's color, shape, etc. The palette type may be determined based on the palette's color or shape.

[0025] Furthermore, the image information of the pallet 3 to be compared may be a template image photographed for each type of pallet 3, or learned image information learned using the image information of the detected pallet 3 may be used.

[0026] In this embodiment, the system is configured to detect information representing the characteristics of the pallet, such as the color and shape of the pallet, from image information of the entire loading platform, including image information of the pallet 3, thereby determining the position of the pallet 3 on the loading platform, the type of pallet 3, and the number of each type of pallet 3.

[0027] When detecting pallets 3 when loading or unloading luggage 2 placed on the loading platform of a transport vehicle 1, etc., there are time and location constraints compared to checking luggage placed in a warehouse, so it is necessary to accurately determine the number of luggage and pallets quickly and accurately using a simpler method.

[0028] The pallet detection device 10 of this embodiment is configured so that the information about the detected pallet can be corrected as needed. In the pallet detection device 10 of this embodiment, the results of pallet detection using image information of the loading platform acquired by the imaging unit are displayed on the pallet detection device 10, and the worker performing the work of detecting the pallet 3 can correct the pallet detection results as needed while displaying the pallet detection results on the pallet detection device 10.

[0029] The pallet detection device 10 displays the pallet detection results and accepts correction input from the operator detecting the pallet 3 to correct the pallet detection results. The pallet information of the pallet 3 detected by the pallet detection device 10 is sent to the pallet management device 20.

[0030] The pallet management device 20 is configured to manage the pallet information transmitted from the pallet detection device 10. The pallet management device 20 learns image information of the pallet using the pallet information transmitted from the pallet detection device 10, thereby generating image information to be used when detecting the pallet, and can provide the generated image information to the pallet detection device.

[0031] In the palette management device 20, learning is performed using palette information corrected in the palette detection device 10, and the palette detection device 10 detects palettes using image information of the learned palette that has been learned using the corrected palette information, thereby enabling more accurate palette detection.

[0032] For example, if a palette detection operator refers to an image in which a palette has been detected and the palette detection results, and determines that an object that is not a palette has been detected as a palette (false detection), or conversely, an image that should have been detected as a palette has not been detected (missed detection), the misdetected image can be learned as an image other than a palette, and the missed-detection image can be learned as an image of a palette, thereby improving the accuracy of detection.

[0033] <Configuration of the pallet detection device> The configuration of the pallet detection device 10 will be described with reference to Fig. 2. Fig. 2 shows an example of functional blocks of the pallet detection device according to the embodiment of the present invention.

[0034] The pallet detection device 10 comprises an imaging unit 11 for acquiring image information of the loading platform including image information of the pallet, a central processing unit 12 for detecting the pallet using the image acquired by the imaging unit 11, a wireless unit 19 for sending and receiving information to and from the pallet management device 20, a memory unit 16 for storing various data such as detected pallet information and pallet images for comparison, an input unit 17 for the operator detecting the pallet to input correction information, and a display unit 18 for displaying the detection results of the pallet 3.

[0035] The pallet detection device 10 uses image information of the loading platform acquired by the imaging unit to display the results of pallet detection on the pallet detection device 10, and while the pallet detection results are displayed on the pallet detection device 10, correction input from the worker detecting the pallet 3 can be accepted to correct the pallet detection results.

[0036] The central processing unit 12 functions as a pallet detection unit 13 that detects pallets using images acquired by the imaging unit 11, a pallet information acquisition unit 14 that acquires pallet information including pallet image information, pallet position information on the loading platform, pallet type information, and pallet number information based on the detection results of the pallet detection unit 13, and a pallet information correction unit 15 that corrects the pallet information acquired by the pallet information acquisition unit 14 based on the input results at the input unit 17.

[0037] The wireless unit 19 functions as a transmitter that transmits palette information to the palette management device 20 and as a receiver that receives the learned palette image from the palette management device 20 .

[0038] The pallet detection device 10 can be configured by a computer equipped with a CPU, a memory, and an interface for connecting to external devices.

[0039] The storage unit 16 stores pallet detection results and image information of the pallets to be compared with the acquired image information of the loading bay. The image information of the pallets 3 includes information representing the characteristics of the pallets, such as the color and shape of the pallets. The image information of the pallets 3 to be compared may be saved as template images taken for each type of pallet 3, or may be saved as learned image information learned using the image information of the detected pallets.

[0040] <Pallet detection results> Fig. 3 is a diagram for explaining detected pallet information in an embodiment of the present invention. Fig. 3 shows an example of the result of detecting pallets 3 of three different colors (red, yellow, blue) in images (A, B, C, D) of a reference position on a loading platform.

[0041] The top diagram in Figure 3 shows an image of the detected pallet 3 and its position on the loading platform. The position of pallet 3 does not need to be an absolute position; it is sufficient if it can represent the relative position of pallet 3 on the loading platform. For example, as shown in the top diagram in Figure 3, it can be represented by an (X,Y) coordinate position with the position A in the reference position image (A, B, C, D) on the loading platform as the origin.

[0042] The top image of Figure 3 shows palette 3 with three colors (Red, Yellow, Blue), as well as palettes that are detected when an image that should have been detected as a palette is not (missed detection palettes), and palettes that are detected when something that is not a palette is detected as a palette (false detection palettes).

[0043] In this embodiment, the pallet information of the detected pallets is displayed on the display unit 18 of the pallet detection device 10, so that the operator detecting the pallets can identify any missed or incorrectly detected pallets from the pallet information displayed on the display unit 18 and make corrections as necessary.

[0044] The bottom diagram in Figure 3 shows the types and numbers of detected palettes. The numbers of palettes 3 with three different colors (red, yellow, and blue) are "6," "5," and "4," respectively. The "yellow" palette contains three false positive palettes, and does not count the number of missed palettes that should have been detected but were not.

[0045] <Correction of palette detection results> In the pallet detection device 10 of this embodiment, the results of detecting a pallet are displayed on the pallet detection device 10 using image information of the loading platform acquired by the imaging unit, and while the pallet detection results are displayed on the pallet detection device 10, correction input from the worker detecting the pallet 3 can be accepted, and the pallet detection results can be corrected.

[0046] The pallet information of the detected pallet in Figure 3 is displayed on the display unit 18 of the pallet detection device 10. The pallet detection device 10 uses image information of the loading platform acquired by the imaging unit to display the results of pallet detection on the display unit 18, and while the pallet detection results are displayed on the display unit 18, correction input from the worker detecting the pallet 3 can be accepted to correct the pallet detection results.

[0047] Figure 4 is a diagram illustrating palette information after correction in an embodiment of the present invention. In the example of Figure 4, a misdetected palette that should have been detected but was not has been corrected to a palette with the color "Blue," and three falsely detected palettes that were detected as palettes with the color "Yellow" have been deleted. As a result of the correction, the number of palettes 3 for the three colors (Red, Yellow, Blue) is "6," "2," and "6," respectively.

[0048] In this embodiment, the detection results of the pallets by the pallet detection device 10 are displayed, and if the detection results are incorrect, the pallet information is corrected as necessary. By quickly verifying the detected pallet image and the pallet detection results and correcting the detection results as necessary, it is possible to quickly and accurately grasp the number of packages and pallets when packages 2 placed on the loading platform of the transport vehicle 1 or the like are being carried in or out.

[0049] The accuracy of detection may be improved by learning palette images using the corrected detection results. For example, if a user refers to an image in which a palette has been detected and the palette detection results, and it is determined that a non-palette has been detected as a palette (false detection), or conversely, an image that should have been detected as a palette has not been detected (missed detection), the accuracy of detection can be improved by learning the falsely detected image as an image other than a palette and the missed detection image as an image of a palette.

[0050] Here, the position information of the pallet on the loading platform, together with the image information of the detected pallet 3, is the subject of learning as information indicating the reliability of the detected image information. For example, in the pallet detection results in Figure 3, the falsely detected pallet is detected at an X coordinate position different from the X coordinate position of the correct pallet on the loading platform, or at a Y coordinate position where pallet 3 would not normally be loaded, and the image of the falsely detected pallet detected in such a position is learned as an image other than a pallet.

[0051] On the other hand, in the pallet detection results in Figure 3, the missed detection pallet is loaded at the Y coordinate position where pallet 3 would normally be loaded, and the image of the missed detection detected at such a position is learned as the image of the pallet.

[0052] 3 and 4 show a case where the pallet detection results are displayed on the display unit 18, and correction input is accepted from the operator detecting the pallet 3, thereby correcting undetected pallets and incorrectly detected pallets. However, it is also possible to set specific positional information in advance, such as the positional information of the pallet on the loading platform, the correct X coordinate position of the pallet on the loading platform, the Y coordinate position where the pallet is loaded, and the Y coordinate position where the pallet is not loaded, and have the pallet information correction unit 15 of the pallet detection device 10 correct undetected pallets and incorrectly detected pallets.

[0053] The dotted line position in Figure 3 is an example of a specific position of pallet 3 on the loading platform. Specific positions (1), (2), and (3) respectively represent the correct X coordinate position of the pallet on the loading platform, the Y coordinate position where the pallet is loaded, and the Y coordinate position where the pallet is not loaded. Even if a pallet is positioned away from specific position (1) or is at specific position (1), the pallet at specific position (3) can be determined to be a falsely detected pallet, and the image of specific position (2) is a position where it should actually be detected as a pallet, so it can be determined to be a missed-detection pallet.

[0054] By correcting the pallet information in the pallet information correction section 15 of the pallet detection device 10 based on the specific position information of the pallet 3 on the loading platform, or by combining the correction input by the worker detecting the pallet 3 with the correction by the pallet information correction section 15 of the pallet detection device 10, it becomes possible to eliminate the need for the worker to input corrections to the pallet information, or to reduce the workload associated with inputting corrections.

[0055] Furthermore, it is also possible to configure the system to detect falsely detected pallets based on differences in their relative positions relative to other pallets, without having to pre-register the correct X-coordinate position of the pallet on the loading platform. For example, the X-coordinate positions of detected pallets 3 may be compared, and a pallet 3 with an X-coordinate position different from the X-coordinate position of the majority of pallets 3 may be determined to be a falsely detected pallet.

[0056] <Operation sequence of the pallet detection system> The palette detection operation in the palette detection system will be described with reference to Fig. 5. Fig. 5 shows an example of the operation sequence of the palette detection system in the embodiment of the present invention.

[0057] The pallet management device 20 acquires image information of the pallet, including information such as the type and shape of the pallet to be detected, and provides this information to the pallet detection device 10 (steps 100, 101).

[0058] The pallet detection device 10 acquires image information (first image information) including an image of a pallet loaded on a loading platform, and detects the image of the pallet from the image information acquired by the imaging unit by comparing the acquired image information with image information of the pallet (second image information) transmitted from the pallet management device 20 and registered in advance (steps 200, 201).

[0059] Based on the detection result of the pallet, the pallet detection device 10 acquires and displays pallet information (first pallet information) including pallet image information, pallet position information on the loading platform, pallet type information, and pallet number information (step 202).

[0060] The pallet detection device 10 corrects the pallet information based on the input results of the correction of the pallet information and the position information of the pallet 3, and transmits the corrected pallet information (second pallet information) to the pallet management device 20 (steps 203, 204).

[0061] The palette management device 20 stores the palette information transmitted from the palette detection device 10, learns the palette image information using the corrected palette information, generates image information for palette detection, and provides the generated learned palette image information to the palette detection device 10 (steps 300, 301, 302).

[0062] The palette detection device 10 uses the learned palette image information provided by the palette management device 20 to detect palette 3, display palette information for the detected palette 3, and correct the palette information, and then transmits the corrected palette information to the palette management device 20 and stores it (steps 400-404, 500).

[0063] In this embodiment, the detection results of the pallets by the pallet detection device 10 are displayed, and if the detection results are incorrect, the pallet information is corrected as necessary and the corrected results are sent to the pallet management device. By quickly verifying the detected pallet image and the pallet detection results and correcting the detection results as necessary, it is possible to quickly and accurately grasp the number of packages and pallets when packages 2 placed on the loading platform of a transport vehicle 1 or the like are being carried in or out.

[0064] The accuracy of palette detection can be improved by learning palette images using the corrected palette detection results and performing palette detection using the learned palette images. For example, if an operator refers to the palette detected image and palette detection results on the spot and determines that there was a palette (missed detection) where an image that should have been detected as a palette was not detected, or a palette (false detection) where something that is not a palette was detected as a palette, the image that was missed detection can be learned as a palette image, and the falsely detected image can be learned as an image other than a palette, and palette detection can be performed using the learned results, thereby improving detection accuracy.

[0065] <Operation flow of the pallet detection device> The palette detection operation in palette detection device 10 will be described with reference to Fig. 6. Fig. 6 shows an example of the operation flow of the palette detection device according to the embodiment of the present invention.

[0066] The pallet detection device 10 acquires image information of the pallet that includes information such as the type and shape of the pallet to be detected (step S1-1).

[0067] The pallet detection device 10 acquires the reference position of the loading platform, and acquires image information (first image information) including an image of the pallet loaded on the loading platform (steps 1-2 and 1-3).

[0068] The pallet detection device 10 detects the image of the pallet from the image information acquired by the imaging unit by comparing the image information acquired by the imaging unit with the image information of the pallet (second image information) transmitted from the pallet management device 20 and registered in advance (step S1-4).

[0069] Based on the detection result of the pallet, the pallet detection device 10 acquires pallet information (first pallet information) including image information of the pallet, position information of the pallet on the loading platform, type information of the pallet, and information on the number of pallets, and displays the acquired results (step S1-5).

[0070] The pallet detection device 10 corrects the pallet information based on the input results of the correction of the pallet information and the position information of the pallet 3, and transmits the corrected pallet information (second pallet information) to the pallet management device 20 (steps S1-6, S1-7).

[0071] Palette management device 20 learns the palette image based on the corrected palette information. Palette detection device 10 receives palette image information learned based on the corrected palette information from palette management device 20, and performs palette detection using the learned palette image information in a flow similar to the above-described operational flow (steps S1-1 to S1-7).

[0072] As described above, the pallet detection system of this embodiment makes it possible to quickly and accurately identify pallets loaded on the loading platform of a transport vehicle using a simpler method, even when time or location is limited. Furthermore, by quickly verifying the detected pallet image and the pallet detection results and correcting the detection results as necessary, the number of pallets can be quickly and accurately determined. Furthermore, the corrected pallet detection results are used to learn the pallet images, and the learned pallet images are used to perform pallet detection, thereby improving the accuracy of pallet detection. [Explanation of symbols]

[0073] 1...transport vehicle, 2...baggage, 3...pallet, 10...pallet detection device, 11...imaging unit, 12...central processing unit, 13...pallet detection unit, 14...pallet information acquisition unit, 15...pallet information correction unit, 16...memory unit, 17...input unit, 18...display unit, 20...pallet management device

Claims

1. A pallet detection system including a pallet detection device that detects pallets on a loading platform, and a pallet management device that manages information about the pallets using the detection results of the pallet detection device, The pallet detection device an imaging unit that acquires first image information including an image of a pallet loaded on a loading platform; a palette detection unit that detects an image of the palette from the first image information by comparing the first image information with second image information of the palette that has been registered in advance; a pallet information acquisition unit that acquires first pallet information including image information of the pallet, position information of the pallet on the loading platform, type information of the pallet, and information on the number of pallets based on a detection result of the pallet detection unit; a palette information correcting unit that corrects the first palette information; a transmitting unit that transmits second pallet information obtained by correcting the first pallet information to the pallet management device; Equipped with Pallet detection system.

2. The palette information correction unit Detecting falsely detected pallets and / or undetected pallets in the first pallet information based on specific position information of the pallet on the loading platform that has been registered in advance, and correcting the first pallet information to the second pallet information. The pallet detection system of claim 1 .

3. The pallet detection device a display unit that displays the first palette information; and an input unit that inputs correction information for the first palette information, The palette information correction unit The first palette information is corrected to the second palette information based on the correction information in the input unit. The pallet detection system of claim 1 .

4. The pallet management device The second image information of the palette is generated by learning image information of the palette using the second palette information, and the generated second image information is provided to the palette detection device. The pallet detection system of claim 1 .

5. The transmission unit of the pallet detection device The pallet information correction unit transmits the first pallet information of the falsely detected pallet and / or the undetected pallet to the pallet management device; The pallet management device The second image information of the palette is generated by learning the second palette information and the first palette information of the falsely detected palette and / or the missed detection palette, and the generated second image information is provided to the palette detection device. The pallet detection system of claim 4 .

Images