Cargo handling determination device, cargo handling support system, and cargo handling device

Abstract

To provide a cargo handling determination device capable of accurately determining whether a loading state on a pallet is good or bad, and a cargo handling support system capable of supporting good cargo handling based on an accurate determination result of the loading state.SOLUTION: A cargo handling determination device includes: an image acquisition unit that acquires an image of a load H on a pallet P; and a control unit that determines whether a loading state is good or bad, wherein the control unit calculates a center-of-gravity position M of the load H based on the image acquired by the image acquisition unit and determines whether the loading state is good or bad based on the calculated center-of-gravity position M.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a cargo handling determination device, a cargo handling support system, and a cargo handling device. [Background technology]

[0002] Patent Document 1 discloses a system that acquires images of a load being transferred by a forklift, and determines that the load has collapsed if the amount of difference in the images over time exceeds a threshold value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-147649 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional cargo collapse detection system, there is a risk that even if the cargo is transferred stably, it may be determined that the cargo has collapsed simply because the cargo has shifted position, and there is a risk that a cargo collapse detection will not be made if the cargo has become less stable but has not shifted position.In addition, there is a problem in that it is difficult to accurately detect the amount of difference in the cargo image over time if there is a change in the background of the cargo.

[0005] The present invention aims to provide a cargo handling judgment device that can accurately judge whether the loading condition on a pallet is good or bad, and a cargo handling support system and cargo handling device that can support good cargo handling based on this judgment. [Means for solving the problem]

[0006] The cargo handling determination device according to the present invention comprises: Multiple loads were loaded On the palette of an image acquisition unit that acquires an image; a control unit that determines whether the loading state is good or bad; Equipped with The control unit performs the following on the basis of the image acquired by the image acquisition unit. Multiple Load individual Center of gravity position , or the individual center of gravity positions and the overall center of gravity position and calculating based on the calculated center of gravity position: , indicating whether the plurality of loads are stably stacked on the pallet. Determine whether the loading condition is good or bad.

[0007] The cargo handling assistance system according to the present invention comprises: The cargo handling determination device; a support device that supports cargo handling based on a determination result of the cargo handling determination device; Equipped with.

[0008] The cargo handling device according to the present invention comprises: The loading and unloading support system is equipped with at least a portion of the image acquisition unit, the control unit, and the support device, and / or communicates with at least a portion of the image acquisition unit, the control unit, and the support device, and performs loading and unloading based on the judgment results of the loading and unloading judgment device. [Effects of the Invention]

[0009] The cargo handling determination device according to the present invention can accurately determine whether the loading condition on a pallet is good or bad. The cargo handling assistance system and cargo handling device according to the present invention can provide support for good cargo handling based on the above determination. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a cargo handling determination device and a cargo handling support system according to the present invention. [Figure 2] 10 is a flowchart showing a determination process executed by a control unit. [Figure 3] FIG. 10 is a diagram illustrating an example of a determination region. [Figure 4] 10 is a flowchart showing a cargo handling support process executed by the support device. [Figure 5] 10A and 10B are diagrams illustrating the loading state determination process of the first modified example. [Figure 6] 10A and 10B are diagrams illustrating the loading state determination process of the second modified example. [Figure 7] 13A is a diagram of a first example illustrating a loading state determination process according to Modification 3, and FIG. 13B is a diagram of a second example illustrating the loading state determination process according to Modification 3. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings.

[0012] <Cargo handling determination device> FIG. 1 is a block diagram showing a cargo handling determination device and a cargo handling support system according to the present invention.

[0013] The cargo handling determination device 100 of this embodiment is a device that determines whether the loading condition of cargo H on a pallet P held and transported by a cargo handling vehicle 30 serving as a cargo handling device is good or bad. The cargo handling vehicle 30 is a vehicle such as a forklift that performs cargo handling (transporting, loading, unloading, etc.) via the pallet P. The pallet P is a platform on which the cargo H is loaded, and has a structure e (for example, a horizontal hole into which the forks f of a forklift can be inserted) that can be held by the cargo handling vehicle 30. Multiple cargoes H can be loaded on the pallet P.

[0014] The cargo handling determination device 100 includes an image acquisition unit 110 that acquires images of the cargo H on the pallet P, and a control unit 120 that determines whether the loading condition is good or bad. The control unit 120 calculates the position M (FIG. 3) of the center of gravity of the cargo based on the image acquired by the image acquisition unit 110, and determines whether the loading condition is good or bad based on the calculated position M (FIG. 3) of the center of gravity of the cargo. The quality of the loading condition is an index that indicates whether the cargo H is stably loaded on the pallet P, that is, whether there is a high or low risk of cargo collapse when the cargo handling vehicle 30 transports the cargo H via the pallet P.

[0015] The load H to be subjected to the load condition determination is a load H whose center of gravity position M can be determined from its shape. The shape of the load H may be of one type or of multiple types. The shape of the load H to be subjected to the load may be a box representing the outer shape of the load H containing items under predetermined conditions, and the shape of the load H may be associated with the center of gravity position M. Alternatively, data indicating the correspondence between the shape of the load H and the center of gravity position M may be provided to the control unit 120 in advance, and the shape of the load H may be associated with the center of gravity position M based on this data.

[0016] The image acquisition unit 110 acquires an image of the load H on the pallet P. The image acquisition unit 110 may be a digital camera having a lens and an image sensor, in which case the image will be an RGB video. Alternatively, the image acquisition unit 110 may be a scanner device that acquires two-dimensional or three-dimensional shapes by scanning light or sound waves and detecting reflected waves. In this case, the image will be a scanner image. The image acquisition unit 110 may include a distance sensor, and the acquired image may include distance information. The distance sensor may be any device that can measure the distance between multiple points in an image, such as a LiDAR (Light Detection and Ranging) or a compound eye camera.

[0017] The image acquisition unit 110 may be configured to acquire images of the load on the pallet P from one direction, or may be configured to acquire images of the load on the pallet P from multiple directions.

[0018] The image acquisition unit 110 may be a first image acquisition unit 110a mounted on the cargo handling vehicle 30, or a second image acquisition unit 110b installed at a location separate from the cargo handling vehicle 30, or may be a combination of these.

[0019] The image acquisition unit 110 transmits image data to the control unit 120 via wired or wireless communication.

[0020] The control unit 120 is a computer equipped with a CPU (Central Processing Unit) that executes a program. The control unit 120 has a storage unit that stores a determination processing program 122 and control data 123. The control data 123 includes data indicating a determination region R, which will be described later. The control unit 120 further has a machine-learned learning model 121. The learning model 121 may be an AI (Artificial Intelligence) having a deep-learned neural network. A plurality of teacher data for a plurality of samples is provided in advance to the learning model 121, and the learning model 121 is machine-learned using the plurality of teacher data.

[0021] <Determination process> Fig. 2 is a flowchart showing the determination process executed by the control unit 120. Fig. 3 is a diagram showing an example of the determination region.

[0022] The control unit 120 executes the determination process based on a predetermined start condition. The start condition may be set appropriately to indicate the timing at which the loading status should be confirmed, such as immediately before the loading vehicle 30 starts loading or during loading (for example, when passing in front of the second image acquisition unit 110b installed in the loading area).

[0023] When the determination process begins, the control unit 120 receives an image acquired by the image acquisition unit 110 (step S1). Then, upon receiving the image, the control unit 120 calculates the center of gravity M of the load H included in the image (step S2). Specifically, the control unit 120 calculates the center of gravity M of each of the multiple loads H on the side from which the image was acquired. Then, the control unit 120 determines whether the loaded state is good or bad based on the calculated center of gravity M and the determination region R (step S3). Specifically, the control unit 120 determines whether all of the calculated center of gravity M are within the determination region R (step S3), determines that the loads H on the pallet P are stable and in a good loaded state, and transmits the determination result to the support device 210 (step S4), which will be described later. On the other hand, if one or more of the calculated center of gravity positions M are not within the judgment area R, the control unit 120 determines that the load H on the pallet P is unstable and that the loading condition is poor, and transmits the judgment result to the support device 210 described below (step S5).

[0024] In the calculation process of step S2, the control unit 120 may calculate the center of gravity position M of the multiple loads H using the learning model 121. The learning model 121 may be provided in advance with multiple pieces of training data indicating images of multiple samples and the outlines of each load H included in the samples, and the learning model 121 may be machine-learned using the multiple pieces of training data. In this case, the control unit 120 inputs the image acquired by the image acquisition unit 110 into the learning model 121, and information on the outlines of the multiple loads H in the image is output from the learning model 121. The dashed dotted lines in FIG. 3 indicate an example of the outlines extracted by the control unit 120 using the learning model 121.

[0025] Once the information on the outline is obtained, the control unit 120 calculates the center of gravity position M of multiple loads H by assuming that a heavy object is contained within the load H surrounded by the outline under specified conditions, or calculates the center of gravity position M of multiple loads H by referring to data showing the correspondence between the shape of the load H and the center of gravity position M.

[0026] The above outline may be interpreted as the circumscribing line of load H or the inscribing line of load H. The circumscribing line refers to the smallest rectangle circumscribing load H in the image (a rectangle enclosed by vertical and horizontal lines in the image), and the inscribing line refers to the largest rectangle inscribing load H in the image (a rectangle enclosed by vertical and horizontal lines in the image). When load H is photographed from the front or from the side, the outline of load H and the circumscribing line or inscribing line nearly coincide. Therefore, in this case, even if the circumscribing line or inscribing line is used, the center of gravity position M of load H can be calculated in a similar manner. Furthermore, when load H is photographed from a position close to the front or from the side, the outline of load H and the circumscribing line or inscribing line are similar. Therefore, even in this case, even if the circumscribing line or inscribing line is used, the center of gravity position M of load H can be calculated within the allowable error using a similar process.

[0027] Alternatively, the learning model 121 may be provided in advance with a plurality of training data indicating images of a plurality of samples and the center of gravity positions M of each load H included in the samples, and the learning model 121 may be machine-learned using the plurality of training data. In this case, the control unit 120 inputs the images acquired by the image acquisition unit 110 to the learning model 121, thereby causing the learning model 121 to output the center of gravity positions M of the plurality of loads H in the images.

[0028] By calculating the center of gravity position M using the learning model 121, it is possible to suppress erroneous recognition of the outline of the load H due to the background of the load H on the pallet P, or erroneous calculation of the center of gravity position M, and it is possible to realize accurate calculation of the center of gravity position M with little effort. Furthermore, by calculating the center of gravity position M using the learning model 121, even if an inexpensive photographing device is used as the image acquisition unit 110, it is possible to realize accurate calculation of the center of gravity position M based on the detection results.

[0029] Additionally, the control unit 120 may perform data analysis (such as image analysis) on the image acquired by the image acquisition unit 110 to extract the outline of the load H, and calculate the center of gravity position M of the load H from the outline.

[0030] In step S2, the control unit 120 may determine the center of gravity positions M only for the multiple loads H located on the image acquisition side, rather than for all of the multiple loads H stacked on the pallet P. Even in this case, the subsequent determination process in step S3 can accurately determine the stability of the multiple loads H located on the image acquisition side of the pallet P.

[0031] Furthermore, in step S2, the control unit 120 may determine the center of gravity position M as a position on two-dimensional coordinates as viewed from the image capture side, rather than as a position on three-dimensional coordinates. Even in this case, the subsequent determination process in step S3 allows the stability of multiple loads H on the pallet P in the two-dimensional directions to be accurately determined.

[0032] The example in FIG. 3 is an example in which the control unit 120 determines the center of gravity positions M as positions on two-dimensional coordinates viewed from the image acquisition side only for a plurality of loads H located on the image acquisition side.

[0033] The judgment area R is set so that it is the boundary between whether the load H is stable or unstable during loading via the pallet P, depending on whether the center of gravity position M of the load H is inside or outside the area. The judgment area R in Fig. 3 is used to determine whether the loaded state is good or bad based on the center of gravity positions M of the individual loads H, and is set so that the boundary is located inside by a margin from the edge of the pallet P. The judgment area R may also be set so that the boundary is linear in the height direction.

[0034] <Setting the judgment area> As shown in FIG. 1, the cargo handling determination device 100 may further include a setting unit 130 capable of setting a determination region R for the center of gravity position M. The setting unit 130 includes a display device 131 that outputs a setting display and an input device 132 that specifies the determination region R. The administrator can specify the determination region R via the input device 132 while viewing the display on the display device 131. The display device 131 and the input device 132 may not be included in the cargo handling determination device 100, but may be a display device and an input device included in another computer or another mobile terminal. In this case, the setting unit 130 of the cargo handling determination device 100 may be software that communicates with the other computer or another mobile terminal, outputs a setting display, and performs input processing to specify the determination region R. The administrator can, for example, narrow or widen the determination region R in FIG. 3. Alternatively, the administrator can change the setting to increase, decrease, or change the shape of the determination region R in FIG. 3. With this configuration, the administrator can change the setting of the judgment area R to suit various situations, such as situations where a very high level of cargo handling stability is required and situations where a relatively low level of cargo handling stability is required.

[0035] <Cargo handling support system> As shown in FIG. 1 , the cargo handling assistance system 200 of this embodiment includes a cargo handling determination device 100 and an assistance device 210 that assists in cargo handling based on the determination result of the cargo handling determination device 100. The assistance device 210 may include a notification unit 211 that notifies the result of the cargo handling determination, and an operation control mechanism 212 that controls the operation of the cargo handling vehicle 30 (such as temporary stop control). The notification unit 211 may be mounted on the cargo handling vehicle 30, or may be located both on the cargo handling vehicle 30 and at the cargo handling site. The notification unit 211 notifies the result of the determination by the cargo handling determination device 100 by audio output, lamp display, image or text display. The notification unit 211 may include a display unit 211a that displays video. The operation control mechanism 212 is, for example, a mechanism that can temporarily suspend or slow down the lifting and lowering of the forks, or a mechanism that can temporarily suspend or slow down the travel of the cargo handling vehicle 30. More specifically, the driving control mechanism 212 is a mechanism that can prohibit the forward movement of the cargo handling vehicle 30 and the insertion of the forks into the pallet P. The driving control mechanism 212 may be mounted on the cargo handling vehicle 30, or, if the cargo handling vehicle 30 is an automatic driving vehicle, may be mounted on a device that outputs instructions for automatic driving.

[0036] FIG. 4 is a flowchart showing the cargo handling support process executed by the support device.

[0037] As shown in the above-described determination process (FIG. 2), the control unit 120 of the cargo handling determination device 100 transmits the determination result of whether the loading condition is good or bad to the support device 210. The support device 210 receives the determination result (step S11), discriminates the determination result (step S12), and, if the determination result is good, executes an operation for determining a good loading condition (step S13). Specifically, in step S12, the notification unit 211 outputs a lamp of a color indicating a good determination, and outputs text, an image, and a sound indicating a good determination. Furthermore, as an operation for determining a good loading condition, the support device 210 may output, on the display unit 211a, an image in which an image of the load H on the pallet P, a dot image indicating the calculated center of gravity position M, and a pattern representing the determination region R are superimposed.

[0038] Furthermore, if the determination result is an improper loading condition, the support device 210 executes an operation for determining a fault (step S14). Specifically, in step S14, the notification unit 211 outputs a lamp in a color and flashing motion indicating a fault determination, and outputs text, an image, and a sound indicating a fault determination. Furthermore, as an operation for determining a fault, the support device 210 may output, to the display unit 211a, an image in which a video of the load H on the pallet P, a dot image indicating the calculated center of gravity position M, and a pattern representing the determination region R are superimposed. By viewing this image, the loading worker can recognize which load H is poorly positioned. Furthermore, in step S14, the operation control mechanism 212 performs operation control to temporarily halt or slow down the operation of the loading vehicle 30.

[0039] As described above, according to the cargo handling determination device 100 and cargo handling assistance system 200 of this embodiment, the cargo handling determination device 100 determines whether the loading condition is good or bad based on the center of gravity position M of the cargo H, so that it is possible to make a determination that accurately reflects the stability of the cargo H. Then, based on the determination result, the assistance device 210 assists in cargo handling, so that it is possible to realize assistance processing that accurately reflects the quality of the loading condition.

[0040] (Variation 1) Fig. 5 is a diagram illustrating the loading state determination process of Modification 1. Fig. 5 shows the image acquisition unit 110 and the load H on the pallet P viewed from the side.

[0041] The first modification is different from the above embodiment in the calculation of the center of gravity position Ma by the control unit 120 and in part of the method for determining whether the loading condition is good or bad, but is otherwise similar to the above embodiment.

[0042] In variant example 1, in step S2 of the judgment process in Figure 2, in addition to the calculation process of the center of gravity position M described above, the control unit 120 calculates the center of gravity position Ma in the depth direction as seen from the side where the image is acquired for multiple loads H stacked on the pallet P that are located on the side where the image is acquired.

[0043] Specifically, the control unit 120 calculates the depthwise position of each load H based on information about the distance to each load H (loads H located on the image acquisition side) shown in the image and information about the distance to the pallet P. The depth dimensions and depthwise shape of each load H may be provided to the control unit 120 in advance, or the control unit 120 may determine the depth dimensions and depthwise shape of each load H based on images acquired by the second image acquisition unit 110b, which acquires images of the load H from the side of the pallet P. After calculating the depthwise position of each load H, the control unit 120 calculates the depthwise center of gravity position Ma of each load H based on the calculated position. For example, the distance sensor of the image acquisition unit 110 may measure only the distance to the point of the center of gravity position M as seen from the image acquisition side for each load H whose center of gravity position M is to be calculated. This configuration further reduces the calculation load for the depthwise center of gravity position Ma.

[0044] In the first modification, the control data 123 stored in the control unit 120 further includes a determination area Ra for determining whether the depth-wise center-of-gravity position Ma is acceptable. The determination area Ra is set so that it defines the boundary between whether the load H is stable or unstable during loading via the pallet P, depending on whether the depth-wise center-of-gravity position Ma of the load H is inside or outside the area. The determination area Ra in Fig. 5 is used to determine whether the loaded state is acceptable based on the individual center-of-gravity positions Ma of multiple loads H, and is set so that the boundary is located inside the front end of the pallet P by a margin. The boundary of the determination area Ra may be set so that it is linear in the height direction.

[0045] In the first modification, further, in step S3 of the determination process in Fig. 2, the control unit 120 determines whether the loading condition is good or bad based on the depth-direction center-of-gravity position Ma of each load H and the depth-direction determination area Ra, in addition to the determination elements of step S3 described above. Specifically, the control unit 120 determines that the loading condition is good if the depth-direction center-of-gravity position M of each load H located on the image acquisition side falls within the determination area R and the depth-direction center-of-gravity position Ma of each load H falls within the determination area Ra. Furthermore, the control unit 120 determines that the loading condition is bad if the depth-direction center-of-gravity position M of each load H when viewed from the image acquisition side falls outside the determination area R, or if the depth-direction center-of-gravity position Ma of each load H falls outside the determination area Ra.

[0046] In addition to the setting process described above, the setting unit 130 (FIG. 1) may be configured to be able to change the setting of the determination region Ra in the depth direction. In the setting change process of the determination region Ra, the position, size, and shape of the determination region Ra may be changeable.

[0047] The cargo handling determination device 100 of the first modification can make a determination that accurately reflects the stability of the cargo H in the depth direction as viewed from the image acquisition side. The cargo handling assistance system 200 of the first modification performs cargo handling assistance based on the above-mentioned determination results, so that assistance processing that accurately reflects the quality of the loading condition can be realized.

[0048] (Variation 2) FIG. 6 is a diagram illustrating the loading state determination process of the second modification.

[0049] The second modification is different from the above embodiment and the first modification in the calculation of the center of gravity by the control unit 120 and in part of the method of determining whether the loading condition is good or bad, but is otherwise similar to the above embodiment or the first modification.

[0050] In Modification 2, in step S2 of the determination process in FIG. 2, the control unit 120 calculates the overall center of gravity position Mb of the multiple loads H in addition to the calculation process described above. The multiple loads H may not be all loads H on the pallet P, but may be multiple loads H located on the side where the image was acquired. Furthermore, the center of gravity position Mb may be a position in a two-dimensional direction as viewed from the side where the image was acquired. The overall center of gravity position Mb can be calculated from the arrangement of the multiple loads H and the weight ratio of each load H. Alternatively, if the multiple loads H are the same item, the overall center of gravity position Mb can be calculated from the respective center of gravity positions M of the multiple loads H. The weight ratio of each load H, or the fact that each load H is the same item, may be provided to the control unit 120 in advance.

[0051] In the second modification, the control data 123 stored in the control unit 120 further includes a judgment region Rb for judging whether the overall center of gravity position Mb is acceptable. The judgment region Rb is set so that it defines the boundary between whether the loads H are stable or unstable during loading via a pallet P, depending on whether the overall center of gravity positions Mb of multiple loads H are inside or outside the region. The judgment region Rb in Fig. 6 corresponds to the region inside the dashed line. The judgment region Rb in Fig. 6 is set so that it becomes wider as it is lower and narrower as it is higher.

[0052] In Modification 2, in step S3 of the determination process in Fig. 2, the control unit 120 determines whether the loading condition is good or bad based on the overall center of gravity position Mb of the multiple loads H and its determination region Rb, in addition to the determination elements of step S3 described above. Specifically, the control unit 120 determines that the overall center of gravity position Mb is stable if the overall center of gravity position Mb of the multiple loads H falls within the determination region Rb, and determines that the overall center of gravity position Mb is unstable if it does not fall within the determination region Rb. Then, the control unit 120 comprehensively determines whether the loading condition is good or bad based on the determination results of the individual center of gravity positions M of the multiple loads H, or in addition, the center of gravity position Ma in the depth direction.

[0053] In addition to the setting process described above, the setting unit 130 (FIG. 1) may be configured to be able to change the setting of the determination region Rb for the overall center of gravity position Mb. In the setting change process for the determination region Rb, the shape of the determination region Rb may be defined as a triangle or a trapezoid, and the height and length of the upper side may be changeable.

[0054] According to the cargo handling determination device 100 of the second modification, the quality of the loading condition is further determined based on the overall center of gravity position Mb of the multiple loads H, making it possible to make a determination that more accurately reflects the stability of the loads H. According to the cargo handling assistance system 200 of the second modification, cargo handling assistance is performed based on the above-mentioned determination results, making it possible to realize assistance processing that accurately reflects the quality of the loading condition.

[0055] (Variation 3) FIG. 7A is a diagram of a first example and FIG. 7B is a diagram of a second example illustrating the loading state determination process of the third modified example.

[0056] The third modification is different from the above embodiment and modifications 1 and 2 in the calculation of the center of gravity by the control unit 120 and part of the method of determining whether the loading condition is good or bad, but is otherwise similar to the above embodiment or modifications 1 and 2.

[0057] As shown in Figure 7(A), the shape of the load H on the pallet P may be of multiple types. The control unit 120 can calculate the center of gravity position Mc of loads H of different shapes using data that indicates the correspondence between the shape and the center of gravity position, which is provided to the control unit 120 in advance. The outline of the load H (the outline as seen from the side where the image is acquired) can be extracted using the learning model 121. In Figure 7(A), the outline extracted using the learning model 121 is shown by a dashed line.

[0058] 7(B), the load on the pallet P may be a pallet Pa to be transported. In this case, as in the case of the load H having a different shape described above, data indicating the correspondence between the shape of the pallet Pa and the center of gravity position Md is provided in advance to the control unit 120, and the control unit 120 can use this data to determine the center of gravity position Md of the load, which is the pallet Pa to be transported. In FIG. 7(B), the outline extracted using the learning model 121 is shown by a dashed line.

[0059] Alternatively, a plurality of pieces of training data showing loads H or pallets Pa of a plurality of shapes and their center of gravity positions Mc, Md as samples may be provided in advance to the learning model 121, and the learning model 121 may be machine-trained using the plurality of pieces of training data. In this case, the control unit 120 can use the learning model 121 to determine the center of gravity positions Mc of loads H of a plurality of shapes and the center of gravity position Md of the pallet Pa to be transported.

[0060] In Modification 3, the control data 123 stored in the control unit 120 may further include a plurality of judgment regions R1, R2 according to the type of load H. The judgment regions R1, R2 according to the type of load H may be used to judge the acceptability of individual center-of-gravity positions Mc, Md, may be used to judge the acceptability of the center-of-gravity position in depth, or may be used to judge the acceptability of the overall center-of-gravity position.

[0061] In Modification 3, in step S3 of the determination process in Fig. 2, the determination regions R1 and R2 are switched depending on the type of load H, and the determination process is performed using the switched determination region R1 (or R2). The control unit 120 can switch between the determination regions R1 and R2 based on data that indicates the correspondence between multiple types of load H (multiple types of shapes of load H) and multiple determination regions R1 and R2, which data is provided to the control unit 120 in advance. Alternatively, the driver of the loading vehicle 30, other loading workers, or a manager may switch between the determination regions R1 and R2 by operating a button or the like.

[0062] In addition to the setting process described above, the setting unit 130 (FIG. 1) may be configured to be able to change the settings of multiple determination regions R1, R2 according to the type of load H. For example, as shown in FIG. 7(A), if the load H has a high center of gravity Mc and is biased, a narrower determination region R1 can be set. Also, as shown in FIG. 7(B), if the load (pallet Pa to be transported) is wide, a narrower determination region R2 can be set.

[0063] The cargo handling determination device 100 of the third modification can perform a determination that more accurately reflects the stability of the cargo H, even when multiple types of cargo H with different shapes are loaded on the pallet P. The cargo handling support system 200 of the third modification performs cargo handling support based on the above-mentioned determination results, and can therefore perform support processing that accurately reflects the quality of the loading condition, even when multiple types of cargo H with different shapes are loaded on the pallet P.

[0064] The above describes an embodiment of the present invention. However, the present invention is not limited to the above embodiment. For example, in the above embodiment, a forklift was used as an example of the loading device. However, the loading device may be various heavy machinery used to transport, load, and unload loads. Furthermore, the loading device may be operated by a driver or automatically. The pallet on which the load is loaded is not limited to a pallet for a forklift, but may be any platform for transporting loads and transported by heavy machinery. Furthermore, in the above embodiment, an example was shown in which an image of the load H on the pallet P was acquired from the side. However, an image of the load H on the pallet P may be acquired from above, and the quality of the loading condition may be determined based on the center of gravity (center of gravity in the planar direction) of the upper load H. Furthermore, FIG. 1 illustrates an example in which the control unit 120 and the setting unit 130 are installed in a location separate from the loading vehicle 30, and a part of the image acquisition unit 110 (first image acquisition unit 110a) and a part of the support device 210 are mounted on the loading vehicle 30. However, part or all of the control unit 120 and the setting unit 130 may be mounted on the cargo handling vehicle 30. In this case, the parts of the control unit 120, the setting unit 130, and the image acquisition unit 110 mounted on the cargo handling vehicle 30 and the parts installed at a location other than the cargo handling vehicle 30 can communicate with each other to perform similar processing. Also, part or all of the control unit 120 and the setting unit 130 may be located in a cloud (server device) and communicate with the remaining parts, as well as the image acquisition unit 110 and the support device 210, to perform similar processing. Also, the control unit 120 and the setting unit 130 may be divided and mounted on a cloud, a mobile terminal, or a computer on the cargo handling vehicle 30, and these may communicate with each other to perform similar processing. Other details shown in the embodiments may be modified as appropriate without departing from the spirit of the invention. [Explanation of symbols]

[0065] 30 cargo handling vehicles 100 Cargo handling determination device 110 Image acquisition unit 110a 1st image acquisition unit 110b Second image acquisition unit 120 control section 121 Learning Model 130 Setting section 200 Cargo Handling Support System 210 Support equipment 211 Information Department 211a Display section 212 Operation control mechanism P, Pa Palette H load R, R1, R2, Ra, Rb judgment area M, Ma~Md Center of gravity position

Claims

1. An image acquisition unit that acquires an image of a pallet on which multiple loads are loaded; a control unit that determines whether the loading state is good or bad; Equipped with The control unit calculates the center of gravity positions of the plurality of loads individually, or the combination of the individual center of gravity positions and the overall center of gravity position, based on the images acquired by the image acquisition unit, and determines whether the load condition, which indicates whether the plurality of loads are stably stacked on the pallet, is good or bad, based on the calculated center of gravity positions. Cargo handling determination device.

2. the image acquisition unit includes a distance sensor and acquires an image including distance information; The cargo handling determination device according to claim 1.

3. a setting unit capable of setting a region for determining the center of gravity position; the control unit determines whether the item is loaded based on the calculated center of gravity position and the determination area. The cargo handling determination device according to claim 1 or 2.

4. The setting unit is capable of setting at least one of a determination area for determining the position of the center of gravity of each of the plurality of articles on the pallet, a determination area for determining the position of the center of gravity of the entire plurality of articles on the pallet, and a determination area for determining the position of the center of gravity in the depth direction as viewed from the image acquisition side of the image acquisition unit. The cargo handling determination device according to claim 3.

5. The image acquisition unit includes a first image acquisition unit mounted on a cargo handling vehicle. The cargo handling determination device according to any one of claims 1 to 4.

6. The image acquisition unit includes a second image acquisition unit installed outside the cargo handling vehicle. The cargo handling determination device according to any one of claims 1 to 5.

7. the control unit calculates the center of gravity position using a learning model that has been machine-learned using an image of the load and the outer shape, circumscribing line, inscribing line, or center of gravity position of the load as training data. The cargo handling determination device according to any one of claims 1 to 6.

8. The cargo handling determination device according to any one of claims 1 to 7, a support device that supports cargo handling based on a determination result of the cargo handling determination device; A loading and unloading support system.

9. The assistance device has a display unit that displays the calculated center of gravity position. The cargo handling assistance system according to claim 8.

10. A cargo handling device that is equipped with at least a portion of the image acquisition unit, the control unit, and the support device in the cargo handling support system described in claim 8 or claim 9, and / or that communicates with at least a portion of the image acquisition unit, the control unit, and the support device, and performs cargo handling based on the judgment result of the cargo handling judgment device.

Images