Cargo length measurement system, cargo length measurement method, and cargo length measurement program

Abstract

To uniformly, accurately and easily measure a projecting length from a pallet in cargo without depending on an operator.SOLUTION: In a cargo length measurement system 30, a column 16, a board 18, and exposed portions 12a, 12b of a cargo 12 of a pallet 10 in an input image 22 are identified by image recognition using a learning model 46. A first reference portion P01 and a second reference portion P02 are specified from the column 16 in the input image 22. A reference distance in the input image 22 is obtained from the first reference portion P01 and the second reference portion P02. A projection length in the image in the exposed portions 12a, 12b is obtained in the input image 22. Actual lengths L1, L2 of the exposed portions 12, 12b are obtained.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a cargo length measurement system, a cargo length measurement method, and a cargo length measurement program for measuring the length of a portion of a cargo loaded on a pallet. [Background technology]

[0002] In many cases, cargo is loaded on pallets into the cargo compartment of a truck or the like. The size of the cargo is not uniform, and some building materials, for example, are long and may protrude laterally from the pallet. In addition, when multiple cargoes are loaded, they may protrude vertically. Patent Document 1 proposes creating a loading plan that allows the cargo compartment volume to be used efficiently even when the cargo protruding from the pallet is not uniform in size. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-15574 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in order to prepare the above-mentioned loading plan in advance, for post-facto verification, or for use in simulations, it is desirable to measure, record, and store the length of the cargo protruding from the pallet. This measurement of the protruding length is mainly performed by the loading worker using a unique method at his / her discretion, and is then recorded, transcribed, and stored. This series of processes not only raises concerns about bias and errors due to differences in the measuring methods used by each worker, or recording and operation errors, but also places a burden on the workers.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a cargo length measurement system, a cargo length measurement method, and a cargo length measurement program that can uniformly, accurately, and easily measure the protruding length of a cargo from a pallet without the need for an operator. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the cargo length measurement system of the present invention is a cargo length measurement system that measures the protruding length of an exposed portion of the cargo protruding from the edge of a cargo shield of a predetermined second color based on an input image of a pallet having a first reference point and a second reference point separated by a reference distance in a predetermined direction, the pallet having one or more reference elements of a predetermined shape or a predetermined first color, or a combination thereof, and a cargo shield of a predetermined second color, the cargo length measurement system being loaded with cargo of a predetermined third color different from at least the second color, and is characterized by having a memory unit that stores a learning model that has been trained by machine learning based on a plurality of teacher images of the cargo loaded on the pallet, a reference element portion identification means that identifies the reference elements in the input image by image recognition using the learning model, and a cargo identification means that identifies the exposed portion of the cargo in the input image by image recognition using the learning model.

[0007] In addition, the cargo length measurement method of the present invention is a cargo length measurement method that measures the protruding length of an exposed portion of the cargo protruding from the edge of a cargo shield of a predetermined second color based on an input image of a pallet having a first reference point and a second reference point separated by a reference distance in a predetermined direction, the pallet having one or more reference elements of a predetermined shape or a predetermined first color or a combination thereof, and a cargo shield of a predetermined second color, the input image being captured showing a state in which the pallet is loaded with cargo of a predetermined third color that is different from at least the second color, and is characterized in that a processor executes the following steps: reading from a memory unit a learning model that has been trained by machine learning based on a plurality of teacher images of the cargo loaded on the pallet; a reference element portion identification step of identifying the reference elements in the input image by image recognition using the learning model; and a cargo identification step of identifying the exposed portion of the cargo in the input image by image recognition using the learning model.

[0008] Furthermore, the cargo length measurement program of the present invention is a cargo length measurement program that measures the protruding length of an exposed portion of the cargo protruding from the edge of a cargo shield in a predetermined direction based on an input image of a pallet having a first reference point and a second reference point separated by a reference distance in a predetermined direction, and a cargo shield of a predetermined second color, the pallet being loaded with cargo of a predetermined third color that is different from at least the second color, and is characterized in that the program has a processor execute a step of reading from a memory unit a learning model that has been trained by machine learning based on a plurality of teacher images of the cargo loaded on the pallet, a reference element portion identification step of identifying the reference elements in the input image by image recognition using the learning model, and a cargo identification step of identifying the exposed portion of the cargo in the input image by image recognition using the learning model. Effect of the Invention

[0009] In the cargo length measurement system, cargo length measurement method, and cargo length measurement program according to the present invention, exposed parts of the cargo in an input image are identified by image recognition using a learning model, and a first reference point and a second reference point are specified from the reference elements identified in the input image. Then, a reference distance in the image is obtained from the first reference point and the second reference point, and the actual length of the exposed part is calculated based on these. All that is required for an operator is to take a picture with a camera and send the image, and measurements are not performed by a unique means for each operator, and errors due to recording and transcription are prevented, allowing for uniform, accurate, and easy measurement of the length. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic perspective view of a loaded pallet. [Diagram 2] FIG. 13 is a schematic top view showing how a pallet loaded with cargo is photographed. [Diagram 3] FIG. 1 shows a raw image of a pallet and load taken. [Figure 4] FIG. 1 is a block diagram of a cargo management system including a cargo length measurement system. [Diagram 5] FIG. 2 is a diagram showing a teacher image. [Figure 6] 11 is a flowchart of a cargo length measurement method executed by a processing unit in the cargo length measurement system. [Figure 7] FIG. 10 is a diagram showing a schematic diagram of the results of automatic identification of each area. [Figure 8] 11 is a geometric diagram for explaining a calculation process of a horizontal protrusion length. FIG. [Figure 9] 11 is a geometric diagram for explaining a calculation process of an upward protrusion length. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of a cargo length measurement system, a cargo length measurement method, and a cargo length measurement program according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment.

[0012] First, a description will be given of a pallet 10 and a load 12 that are the objects of measurement in this embodiment. Fig. 1 is a schematic perspective view of a pallet 10 on which a load 12 is loaded.

[0013] As shown in FIG. 1, the pallet 10 has a base 14 on which a load 12 is placed, support posts 16 standing upright from the base 14, and boards (load shields) 18 that support the load 12 from the sides. The base 14 is a slightly long rectangular plate shape that corresponds to the shape of the load 12, and is placed horizontally on the floor. A total of four support posts 16 are provided, one pair on each side (see FIG. 2). A total of two boards 18 are provided, one on each side. Each board 18 is fixed to the pair of support posts 16 and supports the load 12 from the sides. The support posts 16 also support the load 12 from the sides via the boards 18.

[0014] Viewed from the side, the pair of support pillars 16 are located near the ends of the base 14 and are horizontally spaced apart by a horizontal reference distance L01. The pair of support pillars 16 are parallel, and their lengths are the vertical reference distance L02. The support pillars 16 are set to be appropriately long, so that a considerable number of cargoes 12 can be stacked vertically. The support pillars 16 have an appropriate width, are strong enough to support the cargoes 12, and are clearly identifiable when photographed by a camera.

[0015] When viewed from the side, the board 18 has the same width as the base 14 and is slightly higher than the support posts 16, and is capable of supporting the cargo 12 from the side over a wide area while being clearly identifiable in a photograph taken by a camera. The width and height of the board 18 may be set to be somewhat smaller than this, but if the board 18 is provided across at least a pair of support posts 16, it can stably support the cargo 12. Also, if the board 18 has an area at least larger than the pair of support posts 16, it will be easier to identify it in a photographed image.

[0016] The support 16 may have a predetermined shape or a predetermined color, or a combination thereof, so that it can be clearly identified in a photograph. Here, "color" in this application is a combination of hue, saturation, and brightness, and is basically a single color, but may include some specified pattern or mark depending on the design conditions. In each figure, the color of the support 16 is shown with a dark dot background. Also, in order to make the support 16 easier to identify, it is preferable that the base part 14 is a different color from the support 16. In this embodiment, the shape of the support 16 is specified in terms of width and height. The color of the support 16 is also specified, for example, green (first color).

[0017] The board 18 is at least a different color (second color) from the cargo 12, for example, gray. In each drawing, the color of the board 18 is shown by a light dotted background.

[0018] The cargo 12 is covered with, for example, cardboard, and is a common ochre color (third color) for cardboard. The cargo 12 is basically all the same color, but slight color differences due to fading are permitted. It is also permitted for the cargo 12 to have some letters, marks, etc. printed on it. In each drawing, the color of the cargo 12 is shown with a medium-density dot background. When viewed from the side, the board 18 covers most of the cargo 12.

[0019] As described below, the pallet 10 is photographed by a camera, but if an image is taken from only one side, the size, shape, color, etc. of the posts 16 and boards 18 for the other side that is not photographed do not need to be specifically specified.

[0020] The cargo 12 may be, for example, a building material product and may be longer than the base portion 14. In other words, the cargo 12 may protrude in the left-right direction beyond the board 18. The horizontal protruding length of the horizontal exposed portion 12a of the cargo 12 protruding horizontally from the horizontal edge 18a of the board 18 is defined as L1. The horizontal exposed portion 12a may occur on both the left and right sides of the board 18. If the cargo 12 is short, the horizontal exposed portion 12a does not occur.

[0021] A plurality of cargoes 12 can be loaded onto the pallet 10. Depending on the number and size of the cargoes 12, some of them may protrude upward beyond the board 18. The upward protruding length of the upper exposed portion 12b of the cargo 12 protruding upward from the upper edge 18b of the board 18 is defined as L2. Depending on the number and size of the cargoes 12, the upper exposed portion 12b may not occur. Even if neither the horizontal exposed portion 12a nor the upper exposed portion 12b occurs, photographing and predetermined processing may be performed during pre-learning and actual measurement. If there is no protrusion, the protrusion amount is output as "0 mm."

[0022] A plurality of pallets 10 are used as they have common specifications in one or more yards (e.g., factories, warehouses, stores, etc.) where cargo 12 is loaded. As can be seen from Figure 1, the pallet 10 has a simple structure. In addition, the colors of the boards 18 and posts 16 can be selected from general-purpose commercially available paints, making it easy to paint or repaint.

[0023] Each system described below uses an image of the pallet 10 loaded with the cargo 12. Figure 2 is a schematic top view showing how the pallet 10 loaded with the cargo 12 is photographed.

[0024] In FIG. 2, if an image is taken from a position close to the pallet 10 on the left-right center C as shown by the reference symbol 20x, the short load 12 on the front side (reference symbol 12A in FIG. 2) and the long load 12 on the back side (reference symbol 12B in FIG. 2) will be photographed with the same length because the pallet 10 has some depth. Therefore, the image is taken by the camera 20 at a position closer to the side where the horizontal exposure portion 12a exists than the left-right center C of the pallet 10 (for example, on the extension line of the edge 18a) and at a position moderately away from the pallet 10. This makes it possible to suppress errors caused by the depth position where the load 12 is placed. When the horizontal exposure portion 12a protrudes from both the left and right ends of the board 18, the image is taken by the camera 20 at separate positions closer to the left and right. Since the image taken on the left side and the image taken on the right side are processed symmetrically, the image on the left side will be described below as an example.

[0025] FIG. 3 is a diagram showing a raw image 22 of the photographed pallet 10 and the load 12. The information required for the photographed image 22 is the entire pair of support columns 16, the entire left horizontal exposed portion 12a, the most protruding portion of the upper exposed portion 12b, and the left edge 18a of the board 18. The base portion 14 is basically photographed so that it is horizontal in the image 22, but some inclination is allowed. In addition, the inclination of the camera 20 in any direction depending on the skill of the photographer is allowed. Furthermore, inherent distortion due to the lens characteristics of the camera 22 is allowed. In the example shown in FIG. 3, the image is distorted to a certain extent, but such an image is also allowed. Thus, there are no particularly difficult conditions for photographing the image 22, and the image can be easily photographed. The image 22 is photographed, for example, by a worker loading the load in the yard. The model of the camera 20 used for photographing may be different for each yard or each worker. In other words, the image 22 may have some difference in image quality due to differences in the lenses and light receiving elements of each model of the camera 20 used to take the image.

[0026] In this application, the term "camera" refers to a digital camera, including those attached to smartphones, tablet terminals, and notebook computers. In this application, the term "image" refers to a still digital image, but a part of a digital video may be used as a still image. Image 22 is basically a color image.

[0027] Next, the cargo length measurement system 30 according to this embodiment will be described. 4 is a block diagram of a loading management system 32 including the cargo length measurement system 30. In addition to the cargo length measurement system 30, the loading management system 32 has a learning system 34 and a loading plan system 36. The cargo length measurement system 30, the learning system 34, and the loading plan system 36 are not necessarily integrated into a single system, and each may be configured independently, with data being exchanged using a predetermined communication network or recording medium. The cargo length measurement system 30, the learning system 34, and the loading plan system 36 may be included in a single computer.

[0028] The cargo length measurement system 30 has a processing unit 30a and a storage unit 30b, the learning system 34 has a processing unit 34a and a storage unit 34b, and the loading plan system 36 has a processing unit 36a and a storage unit 36b. Each processing unit (processor) may be realized, for example, by having a processing device such as a CPU (Central Processing Unit) read and execute a program, that is, by software, or may be realized by hardware such as an IC (Integrated Circuit), or may be realized by a combination of software and hardware.

[0029] A large number of images 22 taken by the cameras 20 at each yard are supplied to the loading management system 32 via a communication network 38, etc., and are further recorded as input images 22 in the memory unit 34b of the learning system 34 or the memory unit 30b of the cargo length measurement system 30.

[0030] The learning system 34 is a part that performs the learning phase in AI (Artificial Intelligence) such as so-called machine learning, deep learning, etc. In this embodiment, in the learning phase, first, a plurality of teacher images 40 are created from a plurality of input images 22.

[0031] FIG. 5 is a diagram showing a teacher image 40. This teacher image 40 is created based on the input image 22 shown in FIG. 3. That is, the teacher image 40 is obtained by individually identifying and designating the horizontally exposed portion 12a, the upper exposed portion 12b, the pair of support pillars 16, and the board 18 of the cargo 12 in the input image 22. This area designation is obtained by, for example, manually correcting as appropriate using a predetermined area discrimination software. In FIG. 4, the designated areas are surrounded by a thick line and are further individually hatched. Note that, to create the teacher image 40, the input image 22 may be used as is, or may be used after appropriate correction (for example, trimming) as a pre-processing, if necessary.

[0032] In the processing unit 34a of the learning system 34, the learning means 34aa performs learning based on a plurality of teacher images 40, and generates a learning model 46 capable of identifying the pair of pillars 16, the horizontal exposure portion 12a, the upper exposure portion 12b, and the board 18 from the features of each image. By performing learning based on a large number of teacher images 40, the learning model 46 becomes able to identify the area of ​​each part even if each image in the image 22 has some distortion or a difference in shape, color, arrangement, or the like.

[0033] The obtained learning model 46 is supplied to and recorded in the memory unit 30b of the cargo length measurement system 30. The format of the learning model 46 is not limited, and may include, for example, an inference program and various parameters. A known AI model can be applied as the learning means 34aa. After generating the learning model 46, the learning system 34 does not need to be operated in particular, and new input images 22 do not need to be input to the memory unit 34b.

[0034] Although it is said that humans cannot grasp the type of inference that AI performs, it is expected that the fact that it is easy for humans to identify each element by looking at the input image 22 will also contribute to improving the accuracy of identification by AI. Therefore, it is considered preferable that the horizontal exposed portion 12a, the upper exposed portion 12b, the pair of supports 16, and the board 18 have different positions, colors, and shapes. Therefore, the colors and shapes of each part of the pallet 10 are individually specified as described above. However, the shapes of the horizontal exposed portion 12a and the upper exposed portion 12b cannot be specified due to their nature, and the loading mode of the cargo 12 is considered to be somewhat free.

[0035] Next, the cargo length measurement system 30 will be described. The processing unit 30a of the cargo length measurement system 30 has an image input means 30aa, a reference element part identification means 30ab, a cargo identification means 30ac, a board identification means 30ad, a reference point specification means 30ae, a reference distance calculation means 30af, an in-image protrusion length calculation means 30ag, an actual length calculation means 30ah, and a result output means 30ai. These means are executed by the processing unit 30a reading and executing a predetermined program. Among these, the reference element part identification means 30ab, the cargo identification means 30ac, and the board identification means 30ad may be executed by a learning model 46.

[0036] Fig. 6 is a flowchart of a cargo length measurement method executed by the processing unit 30a in the cargo length measurement system 30. The processing unit 30a performs an inference phase in AI in steps S2 to S4 in Fig. 6, and further performs predetermined calculation processing in steps S5 to S8.

[0037] Prior to the processing shown in Fig. 6, one or more images 22 taken in each yard are stored in the storage unit 30b as input images 22. The processing in Fig. 6 may be executed each time an input image 22 is input to the storage unit 30b, or may be executed all at once when a predetermined number of images have been stored, or may be executed every predetermined period (for example, one week).

[0038] 6, the image input means 30aa reads the input image 22 from the storage unit 30b. As with the learning system 34, this input image 22 may be subjected to appropriate pre-processing.

[0039] In step S2, the reference element portion identifying means 30ab identifies a pair of pillars 16 in the input image 22 by image recognition using the learning model 46. In step S3, the cargo identifying means 30ac identifies the horizontally exposed portion 12a and the upwardly exposed portion 12b of the cargo 12 in the input image 22 by image recognition using the learning model 46. In step S4, the reference point specifying means 30ae identifies the board 18 in the input image 22 by image recognition using the learning model 46. Note that, although steps S2, S3, and S4 are divided into individual processes for ease of understanding, in practice they may be performed simultaneously or cooperatively based on the learning model 46.

[0040] 7 is a diagram showing a schematic diagram of the results of automatic classification of each area in steps S2 to S4 based on the learning model 46. The schematic diagram shown in FIG.

[0041] Comparing the schematic diagram shown in Fig. 7 with the teacher image 40 in Fig. 5, the pair of posts 16 and the board 18 are in almost the same position and shape, and there is almost no area identification error. It is presumed that the posts 16 are easily identified because they are characterized by green areas extending almost parallel to each other in the vertical direction at positions slightly spaced apart on the left and right. It is also presumed that the board 18 is easily identified because it is characterized by a wide rectangular gray area that almost surrounds the pair of posts 16. It is also presumed that the fact that the positional relationship between the posts 16 and the board 18 is basically fixed in terms of composition is also utilized as an identification condition.

[0042] In addition, when comparing the schematic diagram shown in FIG. 7 with the teacher image 40 in FIG. 5, the horizontal exposure portion 12a and the upper exposure portion 12b are considerably different in shape in the two images, but the identification error of the region is sufficiently small. This is presumed to be because, as a result of learning based on a large number of teacher images 40, the horizontal exposure portion 12a is presumed to be identifiable as a region that is a different color from the board 18, has the same color as the upper exposure portion 12b, and protrudes leftward from the edge 18a of the board 18. Similarly, the upper exposure portion 12b is presumed to be identifiable as a region that is a different color from the board 18, has the same color as the horizontal exposure portion 12a, and protrudes upward from the edge 18b of the board 18. In other words, it is presumed that the learning model 46 identifies the horizontal exposure portion 12a and the upper exposure portion 12b based on the difference in position and color relative to the pair of posts 16 and the board 18, which are easy to identify. The learning model 46 in this embodiment may be constructed by inverting the training data so that all images extend to the left side, thereby improving the efficiency of machine learning.

[0043] Furthermore, as is clear from Figs. 5 and 7, the board 18 occupies a considerably large area in the image. This board 18 largely conceals the fact that the overall shape of the cargo 12 varies considerably depending on the number and loading state, and also conceals most of the different background at the shooting location, and has the effect of simplifying and standardizing the image in the image. Therefore, it is expected that the learning model 46 will be able to easily distinguish the support 16, the board 18, the horizontal exposed portion 12a, and the upper exposed portion 12b. To achieve this effect, it is desirable for the area of ​​the board 18 to be large, but a suitable effect can be obtained if the board 18 is provided between at least a pair of support columns 16 and is set to be larger than the area of ​​the pair of support columns 16. Also, the image may be photographed with a composition in which each element is included in the very edge of the image frame so that unnecessary information is not included in the image, or background that is considered unnecessary may be removed to some extent by trimming processing.

[0044] Steps S5 to S8 are processes for calculating the horizontal protrusion length L1 of the horizontally exposed portion 12a and the upward protrusion length L2 of the upwardly exposed portion 12b based on the regions identified in steps S2 to S4. L1 and L2 are calculated separately. To avoid complicating the explanation, steps S5 to S8 will be explained separately as a calculation process of the horizontal protrusion length L1 and a calculation process of the upward protrusion length L2. FIG. 8 is a geometric diagram for explaining the calculation process of the horizontal protrusion length L1.

[0045] (Calculation process of horizontal protrusion length L1) To calculate the horizontal protrusion length L1, in step S5, the reference point identification means 30ae identifies a first reference point P01 and a second reference point P02 from a pair of pillars 16, which are reference elements identified in the input image, based on predetermined criteria.

[0046] In this case, the predetermined criterion is to first identify a most protruding portion P11 that protrudes most horizontally to the left in the horizontally exposed portion 12a identified in the image, as shown in Fig. 8. Then, points on the pair of pillars 16 that are at the same height as the most protruding portion P11 are identified as the first reference point P01 and the second reference point P02.

[0047] In principle, since the pair of pillars 16 are parallel, points at any height may be used as the first reference point P01 and the second reference point P02. However, since the camera 20 may be tilted in the elevation direction during shooting, causing the pillars 16 to become non-parallel in the image, by using points at the same height as the most protruding part P11 as the first reference point P01 and the second reference point P02, it is possible to suppress errors caused by distortion in the height direction.

[0048] In step S6, the reference distance calculation means 30af calculates a horizontal reference distance L01x in the input image from the first reference point P01 and the second reference point P02 identified in the input image. Each distance in the image is calculated, for example, in units of pixels. Since the pair of supports 16 are located near both ends of the base 14, they are sufficiently separated horizontally in the image, ensuring a sufficient number of pixels and improving accuracy.

[0049] In step S7, the in-image protrusion length calculation means 30ag calculates the horizontal length L1x of the horizontally exposed portion 12a in the input image. Specifically, a point P12 that is at the same height as the most protruding portion P11 in the horizontally exposed portion 12a along the edge 18a of the board 18 is specified. Then, the distance between P11 and point P12 in the image, that is, the horizontal length L01x, is calculated.

[0050] In step S8, the actual length calculation means 30ah calculates the actual length L1 from the horizontal reference distance L01x in the input image and the horizontal length L1x of the horizontal exposed portion 12a. Here, the ratio R between the actual object and the input image is R=L01 / L01x. As described above, the horizontal reference distance L01 is known. Therefore, the actual length L1 is calculated as L1=R×L1x.

[0051] (Calculation process of upward protrusion length L2) Next, a process for calculating the upward protrusion length L2 will be described. Fig. 9 is a geometric diagram for explaining the process for calculating the upward protrusion length L2.

[0052] To calculate the upward protrusion length L2, in step S5, the reference point identifying means 30ae identifies a first reference point P01 and a second reference point P02 based on a predetermined reference from the support 16, which is the reference element identified in the force image.

[0053] In this case, the predetermined criterion is to specify the upper end and the lower end of one of a pair of pillars 16 identified in the image as a first reference point P01 and a second reference point P02, as shown in FIG.

[0054] In step S6, the reference distance calculation means 30af calculates the vertical reference distance L02x in the input image from the first reference point P01 and the second reference point P02 specified in the input image. The support pillar 16 has an appropriate length and is long in the vertical direction in the image, so that a sufficient number of pixels is secured to improve accuracy.

[0055] In step S7, the in-image protrusion length calculation means 30ag calculates the vertical length L2x of the upper exposed portion 12b in the input image. Specifically, first, the most protruding portion P2 that protrudes most upward in the upper exposed portion 12b identified in the image is identified. Then, a straight line 60 connecting the upper ends of the pair of pillars 16 and a straight line 62 that passes through the most protruding portion P2 and is parallel to the straight line 60 are obtained. An intersection point Px between the extension line connecting P11 and point P12 and the straight line 62 is identified in the image. Furthermore, the distance in the height direction between the intersection point Px and the edge 18b of the board 18 is calculated as the length L2x. Note that if the height of the board 18 may change depending on the amount of cargo 12, etc., the reference for the exposure heights L2 and L2x of the upper exposed portion 12b may be the upper end of the pillar 16, etc., instead of the edge 18b of the board 18. In this embodiment, the board 14 may be treated as a type of cargo whose height is variable, and it is advisable to machine-learn the fact that the height of the board 14 is variable using training images.

[0056] In step S8, the actual length calculation means 30ah calculates the actual length L2 from the reference distance L02x in the input image and the upward protruding length L2x of the upward exposed portion 12b. Here, the ratio R between the actual object and the input image is R=L02 / L02x. As described above, the vertical reference distance L02 is known. Therefore, the actual length L2 is calculated as L2=R×L2x.

[0057] In step 9, data relating to lengths L1 and L2 obtained through the series of processes is output to the loading plan system 36 as protrusion length data 48 (see FIG. 4).

[0058] As shown in Fig. 4, the loading plan system 36 stores the projection length data 48 in the memory unit 36b. The processing unit 36a of the loading plan system 36 has a loading plan creation means 50 and a loading simulation means 52. The loading plan creation means 50 creates a loading plan 54 based on the projection length data 48 and stores it in the memory unit 36b. The loading simulation means 52 performs a loading simulation based on the projection length data 48.

[0059] In this manner, the cargo length measurement system 30, cargo length measurement method, and cargo length measurement program according to the present embodiment identify the exposed parts 12a, 12b of the cargo 12 in the input image 22 by image recognition using the learning model 46, and identify the first reference point P01 and the second reference point P02 from the support 16, which is the reference element identified in the input image 22. Then, the reference distances L01x, L02x in the input image 22 are obtained from the first reference point P01 and the second reference point P02, and the actual lengths L1, L2 are calculated based on these. All that is required of an operator is to take an image using the camera 20 and send the image 22, and measurements are not performed in different ways for each operator, which prevents errors due to recording and transcription, and allows the lengths L1, L2 to be measured accurately and easily.

[0060] Although the cover 18 of the pallet 10 has the function of supporting the load 12, it is not necessarily an essential component from the viewpoint of calculating the length L1 of the exposed portion 12a. That is, even if the cover 18 is not present, the exposed portion 12a can be a shield that serves as a reference for measurement, and if the color of the load 12 is different from that of the support 14, the exposed portion 12a can be identified using the color as a reference. In this case, since the support 16 serves as a load shield that vertically shields part of the load 12, the length L1 of the exposed portion 12a can be determined based on the vertical edge of the support 16.

[0061] The cargo length measurement system of the present invention is a cargo length measurement system that measures the protruding length of an exposed portion of the cargo protruding from the edge of a cargo shield of a predetermined second color based on an input image taken of a pallet having one or more reference elements of a predetermined shape or a predetermined first color, or a combination thereof, and a cargo shield of a predetermined second color, the pallet including a first reference point and a second reference point separated by a reference distance in a predetermined direction, and is loaded with cargo of a predetermined third color different from at least the second color, and is characterized in having a memory unit that stores a learning model that has been trained by machine learning based on a plurality of teacher images of the cargo loaded on the pallet, a reference element portion identification means that identifies the reference elements in the input image by image recognition using the learning model, and a cargo identification means that identifies the exposed portion of the cargo in the input image by image recognition using the learning model.

[0062] The cargo length measurement method of the present invention is a cargo length measurement method that measures the protruding length of an exposed portion of the cargo protruding from the edge of a cargo shield of a predetermined second color based on an input image captured of a pallet loaded with cargo of a predetermined third color different from at least the second color, the pallet including a first reference point and a second reference point separated by a reference distance in a predetermined direction.The method is characterized in that a processor executes the following steps: reading from a memory unit a learning model that has been trained by machine learning based on a plurality of teacher images of the cargo loaded on the pallet; a reference element portion identification step that identifies the reference elements in the input image by image recognition using the learning model; and a cargo identification step that identifies the exposed portion of the cargo in the input image by image recognition using the learning model.

[0063] The cargo length measurement program of the present invention is a cargo length measurement program that measures the protruding length of an exposed portion of the cargo protruding from the edge of a cargo shield of a predetermined second color based on an input image of a pallet having a first reference point and a second reference point separated by a reference distance in a predetermined direction, the pallet having one or more reference elements of a predetermined shape or a predetermined first color, or a combination thereof, and a cargo shield of a predetermined second color, the pallet being loaded with cargo of a predetermined third color that is at least different from the second color, and is characterized in that the program has a processor execute: a step of reading from a memory unit a learning model that has been trained by machine learning based on a plurality of teacher images of the cargo loaded on the pallet; a reference element portion identification step of identifying the reference elements in the input image by image recognition using the learning model; and a cargo identification step of identifying the exposed portion of the cargo in the input image by image recognition using the learning model.

[0064] In such a system, method, and program, image recognition using a learning model is used to identify exposed parts of the cargo in the image, and a first reference point and a second reference point are specified from the support pillar, which is a reference element identified in the image. Then, a reference distance in the input image is obtained from the first reference point and the second reference point, and the actual length is calculated based on these. All that is required of an operator is to take a picture with a camera and send the image, and measurements can be performed accurately and easily, regardless of the operator.

[0065] The present invention is characterized by having a reference location identification means for identifying the first reference location and the second reference location from the reference element identified in the input image based on a predetermined criterion, a reference distance calculation means for calculating the reference distance in the input image from the first reference location and the second reference location identified in the input image, an in-image protrusion length calculation means for calculating the length of the exposed portion in the predetermined direction in the input image, and an actual length calculation means for calculating the actual length of the exposed portion from the reference distance in the input image and the length of the exposed portion in the predetermined direction. In this way, the first reference point and the second reference point are identified from the reference element, and the length of the cargo can be determined more accurately based on the first reference point and the second reference point.

[0066] In the present invention, the reference elements are a pair of supports that protrude upward from horizontally spaced positions on a base on which the cargo is placed and support the cargo laterally. Such supports support the load and are easy to identify on an image.

[0067] In the present invention, the predetermined direction is a horizontal direction, and the first reference point and the second reference point are points at the same height on the pair of pillars. Since the pair of support pillars are spaced apart in the horizontal direction, the reference distance can be set appropriately long. In addition, since the support pillars are easy to identify on the image, it is easy to specify the first and second reference points.

[0068] The present invention is characterized in that the first reference point and the second reference point are points at the same height as the most protruding part in the horizontal direction of the exposed portion. This allows accurate measurements to be made even if the captured image is tilted in the elevation direction and the pair of support columns are not parallel to each other.

[0069] In the present invention, the input image is captured at a position closer to the side where the exposed portion exists than the left-right center of the pallet. This makes it possible to suppress measurement errors caused by differences in the cargo position in the depth direction.

[0070] The present invention is characterized in that the specified direction is the vertical direction, the lengths of the pair of pillars are respectively the reference distances, and the first reference point and the second reference point are the upper end and lower end of the pillars. This makes it easy to identify the first reference location and the second reference location.

[0071] In the present invention, the cargo shield is a board fixed to a pair of the posts and supporting the cargo laterally, the posts have a defined first color, the second color of the board is different from the first color, the board is arranged between the pair of the posts and has a larger area than the pair of the posts. This gives the board a reasonably large area in the image, making it easy to identify.

[0072] The present invention is not limited to the above-described embodiment, and can of course be freely modified without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0073] 10 Pallet, 12 Load, 12a Horizontal exposure portion, 12b Upper exposure portion, 14 Pedestal portion, 16 Support, 18 Board (load shield), 18a, 18b Edge, 20 Camera, 22 Image, Input image, 30 Load length measurement system, 30a, 34a, 36a Processing unit, 30aa Image input means, 30ab Reference element part identification means, 30ac Load identification means, 30ad Board identification means, 30ae Reference point identification means, 30af Reference distance calculation means, 30ag In-image protrusion length calculation means, 30ah Actual length calculation means, 30ai Result output means, 30b, 34b, 36b Memory unit, 32 Loading management system, 34 Learning system, 36 Loading plan system, 40 Teacher image, 46 Learning model, 48 Protrusion length data, L01 Horizontal reference distance, L01x Horizontal reference distance in the image, L02 Vertical reference distance, L02x Vertical reference distance in the image, P01 First reference point, P02 Second reference point

Claims

1. A cargo length measurement system that measures a protruding length of an exposed portion of a cargo that protrudes in a predetermined direction from an edge of the cargo shield based on an input image of a pallet that includes a first reference point and a second reference point that are separated by a reference distance in a predetermined direction, the pallet having one or more reference elements of a predetermined shape or a predetermined first color, or a combination thereof, and a cargo shield of a predetermined second color, the pallet being loaded with cargo of a predetermined third color different from at least the second color, the system comprising: A storage unit that stores a learning model that has been trained by machine learning based on a plurality of training images of the cargo loaded on the pallet; a reference element portion identifying means for identifying the reference element in the input image by image recognition using the learning model; a cargo identification means for identifying the exposed portion of the cargo in the input image by image recognition using the learning model; A cargo length measurement system comprising:

2. a reference location identifying means for identifying the first reference location and the second reference location from the reference element identified in the input image based on a predetermined criterion; a reference distance calculation means for calculating the reference distance in the input image from the first reference location and the second reference location specified in the input image; an in-image protrusion length calculation means for calculating a length of the exposed portion in the input image in the predetermined direction; an actual length calculation means for calculating an actual length of the exposed portion from the reference distance in the input image and a length of the exposed portion in the predetermined direction; 2. The cargo length measurement system according to claim 1, further comprising:

3. 3. The cargo length measurement system according to claim 1 or 2, wherein the reference elements are a pair of pillars which protrude upward from horizontally spaced positions on a base on which the cargo is placed and which support the cargo laterally.

4. The predetermined direction is a horizontal direction, 4. The cargo length measurement system according to claim 3, wherein the first reference point and the second reference point are points at the same height on the pair of support pillars.

5. 5. The cargo length measurement system according to claim 4, wherein the first reference point and the second reference point are points at the same height as the horizontally most protruding part of the exposed portion.

6. 6. The cargo length measurement system according to claim 4, wherein the input image is taken at a position closer to the side where the exposed portion is present than the left-right center of the pallet.

7. The predetermined direction is a vertical direction, The lengths of the pair of pillars are the reference distance, 4. The cargo length measurement system according to claim 3, wherein the first reference point and the second reference point are the upper end and the lower end of the support.

8. The cargo shield is a board fixed to the pair of support posts to support the cargo laterally, the strut is defined by the first color; the second color of the board is different from the first color; 6. The cargo length measurement system according to claim 3, wherein the board is provided across a pair of the posts and has an area larger than that of the pair of the posts.

9. A cargo length measurement method for measuring a length of an exposed portion of a cargo that protrudes from an edge of a cargo shield in a predetermined direction, based on an input image of a pallet having one or more reference elements of a predetermined shape or a predetermined first color, or a combination thereof, and a cargo shield of a predetermined second color, the pallet including a first reference location and a second reference location that are separated by a reference distance in a predetermined direction, the cargo shield being loaded with cargo of a predetermined third color different from at least the second color, the method comprising: A step of reading from a storage unit a learning model that has been trained by machine learning based on a plurality of training images of the cargo loaded on the pallet; a reference element portion identification step of identifying the reference element in the input image by image recognition using the learning model; a cargo identification step of identifying the exposed portion of the cargo in the input image by image recognition using the learning model; A cargo length measuring method characterized in that the processor executes the above steps.

10. A cargo length measurement program that measures a protruding length of an exposed portion of a cargo that protrudes in a predetermined direction from an edge of a cargo shield based on an input image of a pallet that includes a first reference location and a second reference location that are separated by a reference distance in a predetermined direction, the pallet having one or more reference elements of a predetermined shape or a predetermined first color, or a combination thereof, and a cargo shield of a predetermined second color, the pallet being loaded with cargo of a predetermined third color different from at least the second color, the program comprising: A step of reading from a storage unit a learning model that has been trained by machine learning based on a plurality of training images of the cargo loaded on the pallet; a reference element portion identification step of identifying the reference element in the input image by image recognition using the learning model; a cargo identification step of identifying the exposed portion of the cargo in the input image by image recognition using the learning model; A cargo length measurement program that causes a processor to execute the above steps.

Images