Loading goods handling system

Abstract

Load handling system (20) that performs load handling operations with a forklift truck, wherein the load handling system comprises: a first detection unit (21, 31) configured to detect, prior to the commencement of load handling work, a load on a pallet held by a fork of the forklift, together with the pallet as a load state using at least one type of sensor; a first abnormality detection unit (32) configured to determine whether the loading state detected by the first detection unit is abnormal; a second detection unit (34) configured to detect a work situation using a variety of sensor types after the load handling work has started; a second abnormality detection unit (35) configured to determine whether the work situation detected by the second detection unit is abnormal; a storage unit (24) configured to store modeled data in which forklift operating data is linked by an operator with information about the load state captured by the first acquisition unit and information about the work situation captured by the second acquisition unit when the load handling work is performed by manual operation of the forklift, wherein The first abnormality detection unit (32) determines whether the loading state detected by the first detection unit is abnormal, by using the modeled data stored in the storage unit, and The second abnormality detection unit (35) determines whether the work situation detected by the second detection unit is abnormal by using the modeled data stored in the storage unit.

Description

TECHNICAL AREA

[0001] The present disclosure relates to a cargo handling system. BACKGROUND

[0002] For example, Japanese patent publication JP 2005 - 8 367 A describes detecting whether a fork is tilted in a front-to-back direction or whether the height of a fork is displaced with respect to a direction in which the fork is inserted into a fork entry hole of a pallet, based on a plurality of sensors provided at a distal end section and a proximal end section of the fork of a forklift, determining an operating mode to correct the tilt and height of the fork, and inserting the fork into the fork entry hole of the pallet by causing the fork to operate according to the operating mode.Furthermore, JP 2022-123 537 A discloses a vehicle-mounted device that obtains an image captured by a vehicle camera mounted on a forklift transporting a pallet carrying a load. The device maps an area encompassing the forklift's gripper, the load, and the pallet, and captures the load's outline coordinates in the image. Based on the load's outline coordinates relative to the reference axis set in the image, the vehicle-mounted device determines whether the load is positioned in such a way that it deviates from the reference axis. WO 2022 / 190 454 A1 discloses a warehouse management system comprising a status determination unit, an operation log management unit, a control unit, and a user work instructions unit.The state determination unit obtains an operating log that indicates the state of an automated guided vehicle (AGV) performing work in a warehouse and the state of cargo handled by the AGV. A facility operations control unit controls the AGV based on a past operating log. The user work instruction unit then outputs the operating log, stored in an operating log storage unit, to a user terminal belonging to a user. Further prior art is known from WO 2022 / 190 634 A1, JP 2017-19 595 A, and JP H10-53 396 A. SUMMARY

[0003] However, if a load on the pallet shifts laterally, adjacent loads can come into contact with each other, leading to a so-called load shift, in which the loads on the pallet fall over. Even if the probability of load shifts occurring before handling the load is low, there is a possibility that a situation leading to load shifts may arise during handling.

[0004] One of the purposes of the present disclosure is to provide a cargo handling system capable of preventing cargo shifting.

[0005] The problem is solved according to the invention by a load handling system according to claim 1. Further features and advantageous embodiments are shown in the dependent claims.

[0006] One aspect of the present disclosure is a cargo handling system that facilitates cargo handling operations.Performs cargo handling operations with a forklift truck, wherein the load handling system comprises: a first detection unit configured to detect a load on a pallet held by a fork of the forklift truck as a loaded state, together with the pallet, using at least one type of sensor, before the load handling work begins; a first abnormality detection unit configured to determine whether the load state detected by the first detection unit is abnormal; a second detection unit configured to detect a work situation using a variety of sensor types after the load handling work has begun; and a second abnormality detection unit configured to determine whether the work situation detected by the second detection unit is abnormal.

[0007] In such a load handling system, before the load handling operation begins, the load on the pallet held by the forks, along with the pallet itself, is detected as the load condition using at least one type of sensor, and it is determined whether the load condition is abnormal. If the load condition is determined to be abnormal, it is possible to suppress the occurrence of load displacements due to interference or similar factors between adjacent loads by stopping the load handling operation or by performing an operation to prevent load displacements. After the load handling operation has begun, the operating situation is detected using a variety of sensor types, and it is determined whether the operating situation is abnormal.If it is determined that the work situation is abnormal, it is possible to suppress the occurrence of load shifts caused by a situation that leads to load shifting, such as pulling the load, by stopping the load handling work or by performing an operation to prevent load shifts.

[0008] Furthermore, one type of sensor can be a camera that captures an image of the pallet and the load. In such a configuration, the loading status, encompassing the load and the pallet, can be easily and cost-effectively recorded using the camera.

[0009] Furthermore, the variety of sensor types can include a camera that captures an image of the pallet and the load, a microphone that records sounds in the vicinity of the forklift, and a vibration sensor that measures the oscillations or vibrations acting on the forks. In such a configuration, the camera, microphone, and vibration sensor make it possible to detect abnormalities in the work situation that lead to load displacement, such as load dragging and collisions between loads that occur during load handling, over a wide area and with high accuracy.

[0010] According to the invention, the load handling system further comprises a storage unit configured to store modeled data in which operating data of the forklift are linked by an operator with information about the load state detected by the first sensing unit and information about the working situation detected by the second sensing unit. When the load handling work is performed by manual operation of the forklift, the first abnormality detection unit can determine whether the load state detected by the first sensing unit is abnormal by using the modeled data stored in the storage unit, and the second abnormality detection unit can determine whether the working situation detected by the second sensing unit is abnormal by using the modeled data stored in the storage unit.In such a configuration, using the modeled data—linked with information about the forklift's operating data recorded by the operator, the load status recorded by the first sensing unit, and the work situation recorded by the second sensing unit at the time of forklift operation—the system determines whether the load status and work situation are abnormal, based on the modeled result of the operator's actual abnormality assessment and evaluation. It is therefore possible to appropriately determine whether the load status and work situation are abnormal or not.

[0011] Furthermore, the load handling system may include: a fallback plan generation unit configured to generate fallback plan data to prevent load displacement when the first abnormality detection unit determines that the load condition is abnormal, or when the second abnormality detection unit determines that the work situation is abnormal; and a fallback handling control unit configured to control the forklift to perform the load handling work according to the fallback plan data generated by the fallback plan generation unit. In such a configuration, when the load condition or work situation is determined to be abnormal, fallback plan data to prevent load displacement is generated, and the forklift is controlled to perform the load handling work according to the fallback plan data. In this way, the occurrence of load displacement is automatically suppressed.

[0012] Furthermore, the load handling system may also include an avoidance possibility determination unit configured to determine whether a load displacement is avoidable when the first load condition abnormality determination unit determines that the load condition is abnormal, or when the second abnormality determination unit determines that the work situation is abnormal; the avoidance plan generation unit can generate the avoidance plan data when the avoidance possibility determination unit determines that the load displacement is avoidable; and the load handling control unit can control the forklift to stop the load handling work when the avoidance possibility determination unit determines that the load displacement is unavoidable.In such a configuration, if the loading condition or work situation is determined to be abnormal, contingency plan data is generated if the load shift is deemed avoidable. Conversely, the forklift is controlled to stop handling the load if the shift is deemed unavoidable. In this way, contingency plan data is generated only as needed, eliminating the need to create it unnecessarily. This simplifies processing.

[0013] Furthermore, the load handling system may also include a storage unit configured to store modeled data in which forklift operating data is linked by the operator with information about the load state, which is captured by the first acquisition unit, and information about the work situation, which is captured by the second acquisition unit, when the load handling work is performed by manual operation of the forklift, and the avoidance possibility determination unit can determine whether the load displacement is avoidable using the modeled data stored in the storage unit.In such a configuration, using the modeled data, which links the forklift operator's operating data with information about the load status captured by the first sensing unit and information about the work situation captured by the second sensing unit at the time of forklift operation, it is determined whether the load shift is avoidable, based on the result of modeling the operator's actual abnormality detection and assessment. It is therefore possible to appropriately determine whether a load shift is avoidable or not.

[0014] (Furthermore, the load handling system may include a storage unit configured to store modeled data in which forklift operating data is linked by an operator with information about the load state, which is captured by the first sensing unit, and information about the work situation, which is captured by the second sensing unit, when the load handling work is performed by manual operation of the forklift; and the avoidance plan generation unit can generate the avoidance plan data using the information about the load state captured by the first sensing unit and the modeled data stored in the storage unit in a case where the first abnormality determination unit determines that the load state is abnormal, and when it is determined that the load displacement is avoidable,and can generate the evasive action plan data using information about the work situation captured by the second detection unit and the modeled data stored in the memory unit in a case where the second abnormality detection unit determines that the work situation is abnormal and when it is determined that the load displacement is avoidable. In such a configuration, it is possible to generate the evasive action plan data using the modeled data, in which the forklift operator's operating data is linked with the information about the load state captured by the first detection unit and the information about the work situation captured by the second detection unit at the time of forklift operation.The data for the contingency plan is obtained based on the results of modeling the actual abnormality detection and assessment by the operator. In this way, the occurrence of cargo shifts can be suppressed appropriately.

[0015] According to the present disclosure, it is possible to suppress the occurrence of cargo shifts. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a side view illustrating a forklift truck to which a load handling system according to an embodiment of the present disclosure is attached; Fig. 2 is a block diagram illustrating a configuration of the system for handling cargo according to an embodiment of the present disclosure; Fig. Figure 3 is a front view illustrating a situation in which cargo on adjacent pallets obstructs each other; Fig. Figure 4 is a flowchart that describes a process for determining the loading state, which is carried out by a Fig. 2 illustrated control is to be executed; Fig. 5 is a flowchart that shows the process for determining the work situation, which is carried out by the in Fig. 2 illustrated control is to be executed; Fig. 6 is a flowchart that shows the process of contingency plan generation, which is carried out by a contingency plan generation unit of the in Fig. 2 illustrated control is to be executed; and Fig. 7 is a flowchart that shows the process of processing the load handling control unit, which is operated by a Fig. 2 illustrated processing unit of the control system is to be executed. DETAILED DESCRIPTION

[0016] An embodiment of the present disclosure is described in detail below with reference to the drawings.

[0017] Fig. Figure 1 is a side view illustrating a forklift truck on which a load handling system according to an embodiment of the present disclosure is applied. Fig. For example, the forklift 1 is a counterbalance forklift. The forklift 1 comprises a drive unit 2 and a load handling unit 3.

[0018] The driving device 2 comprises a vehicle body 4, front wheels 5, which are a pair of a left and a right drive wheel arranged in a front section of the vehicle body 4, and rear wheels 6, which are a pair of a left and a right steering wheel arranged in a rear section of the vehicle body 4.

[0019] The load handling device 3 comprises a mast 7 attached to a front section of the vehicle body 4, a pair of right and left forks 11 attached to the mast 7 so that they can be moved up and down via a lifting console 8 and hold a pallet 10, a lifting cylinder 12 which moves the forks 11 up and down via the lifting console 8, and a tilting cylinder 13 which tilts the mast 7.

[0020] Pallet 10 is, for example, a flat pallet made of plastic or wood. Pallet 10 has a square or substantially square shape in plan view. A load M is placed on pallet 10. Pallet 10 is provided with a pair of left and right fork holes 10a (see Fig. 3), into which the forks 11 are to be inserted.

[0021] Fig. Figure 2 is a block diagram illustrating a configuration of the system for handling cargo according to an embodiment of the present disclosure. Fig. In the present embodiment, the load handling system 20 is a system that controls the forklift 1 to perform load handling operations by automatically operating the forklift 1. In this case, the load handling system 20 performs the load handling work as picking operations. Picking up the load includes inserting the forks 11 into the fork openings 10a of the pallet 10 on which the load M is loaded (hereinafter referred to as the loaded pallet 10M), raising the forks 11 to lift the loaded pallet 10M, and pulling out the loaded pallet 10M.

[0022] The load handling system 20 is mounted on the forklift 1. The load handling system 20 includes a camera 21, a microphone 22, an accelerometer 23, a storage unit 24, an alarm 25 and a control unit 26.

[0023] Camera 21 is an image sensor that captures an image of pallet 10 and the load M (loaded pallet 10M) located in front of forklift 1 and retrieves the captured data. The loaded pallet 10M located in front of forklift 1 also includes the loaded pallet 10M held by the forks 11. Camera 21 is, for example, mounted on mast 7.

[0024] Microphone 22 is a sound sensor that records the sounds in the vicinity of forklift 1 and collects data on the sound recording. Microphone 22 is, for example, attached to mast 7.

[0025] The accelerometer 23 is a vibration sensor that detects the vibrations acting on the forks 11 of the forklift 1 by measuring the acceleration of the forklift 1 and obtaining vibration data. The accelerometer 23 is, for example, attached to the forks 11.

[0026] The storage unit 24 stores modeled data in which operating data of the forklift 1 are linked by an operator with image data of the loaded pallet 10M acquired by camera 21, noise data acquired by microphone 22, and vibration data acquired by accelerometer 23 when load handling operations are carried out by manually operating the forklift 1. The modeled data consists of data obtained by previously collecting data relating to the determination, assessment, and operation by a human in relation to various load conditions and work situations, and by performing modeling using deep learning or similar methods.

[0027] Alarm 25 is a device that alerts the surroundings by means of a noise or similar if there is a possibility that the load M on pallet 10 may shift.

[0028] The controller 26 consists of a CPU, RAM, ROM, an input / output interface, and the like. The controller 26 includes a load state information reference unit 31, a load state abnormality determination unit 32, an avoidance possibility determination unit 33, a work situation information reference unit 34, a work situation abnormality determination unit 35, an avoidance possibility determination unit 36, a contingency plan generation unit 37, and a load handling control unit 38.

[0029] The load status information reference unit 31 receives image data from camera 21 as load status information for the loaded pallet 10M before the start of the load handling work. The load status of the loaded pallet 10M includes not only the load status of the loaded pallet 10M whose load is to be transferred, but also the load status of the loaded pallet 10M adjacent to the loaded pallet 10M whose load is to be transferred (see Fig. 3) The load condition information reference unit 31 together with the camera 21 forms a first detection unit which, before the start of the load handling work, detects the load M on the pallet 10, which is held by the forks 11 of the forklift 1, together with the pallet 10 as a loaded state, using at least one type of sensor.

[0030] The load state abnormality determination unit 32 uses the modeled data stored in the storage unit 24 to determine whether the load state of the loaded pallet 10M, as determined by the load state information reference unit 31, is abnormal. A load state of the loaded pallet 10M is abnormal if there is a possibility of load displacement occurring due to interference or similar factors between horizontally adjacent loads M. The load state abnormality determination unit 32 is a first abnormality determination unit that determines whether the load state detected by the first detection unit is abnormal.

[0031] If the load condition abnormality determination unit 32 determines that the load condition of the loaded pallet 10M is abnormal, the avoidance possibility determination unit 33 determines, based on the modeled data stored in the storage unit 24, whether a load displacement is avoidable.

[0032] After the start of the load handling work, the unit 34 for acquiring information about the work situation receives the image data from camera 21, the sound data from microphone 22, and the vibration data from accelerometer 23 as information about the work situation. The unit for acquiring information about the work situation 34 represents a second acquisition unit that records the work situation using a variety of sensor types after the load handling work has started, in conjunction with camera 21, microphone 22, and accelerometer 23.

[0033] The work situation abnormality determination unit 35 uses the modeled data stored in the storage unit 24 to determine whether the work situation referenced by the work situation information acquisition unit 34 is abnormal. A work situation is considered abnormal if it can cause cargo displacement due to a situation such as towing cargo M, collision between cargo M, or lifting adjacent cargo M. The work situation abnormality determination unit 35 forms a second abnormality determination unit that determines whether the work situation recorded by the second acquisition unit is abnormal.

[0034] If the work situation abnormality determination unit 35 determines that the work situation is abnormal, the avoidance possibility determination unit 36 ​​determines, based on the modeled data stored in the storage unit 24, whether a cargo displacement is avoidable.

[0035] If the avoidance possibility determination units 33 and 36 determine that a cargo shift is avoidable, the avoidance plan generation unit 37 generates avoidance plan data to avoid the cargo shift using the image data from camera 21, the noise detection data from microphone 22, the vibration data from accelerometer 23, and the model data stored in the memory unit 24. The avoidance plan data will be described in detail later.

[0036] If the avoidance possibility determination units 33 and 36 determine that the load displacement is avoidable, the load handling control unit 38 controls the drive unit 2 and the load handling device 3 to perform the load handling work according to the avoidance plan data generated by the avoidance plan generation unit 37. If the avoidance possibility determination units 33 and 36 determine that the load displacement is unavoidable, the load handling control unit 38 controls the drive unit 2 and the load handling device 3 to stop the load handling work and controls alarm 25 to issue an alarm.

[0037] Fig. Figure 4 is a flowchart that describes the process of determining the load state, which is to be performed by the controller 26. This process is carried out by the load state information reference unit 31, the load state abnormality determination unit 32, and the avoidance possibility determination unit 33. This processing is performed before the load handling work begins in a state where the forklift 1 has reached a position in front of the loaded pallet 10M.

[0038] In Fig. 4. The controller 26 first obtains image data from camera 21 (step S101). Subsequently, the controller 26 assesses whether the current loading state of the loaded pallet 10M is normal, based on the image data from camera 21 (step S102). As in Fig. As illustrated in Figure 3, the loading condition of the cargo is abnormal, for example, if the cargo M on pallet 10 collides with the cargo M on the adjacent pallet 10 (see A in the drawing).

[0039] In this case, a threshold for assessing the abnormality of the loading state is calculated based on the modeled data stored in memory unit 24. For example, if, as in Fig. Figure 3 illustrates that if the overhang d of the load M in relation to the pallet 10 is equal to or greater than the threshold value, there is a risk of load displacement occurring, and therefore the loading condition is judged to be abnormal.

[0040] If the current loading state of pallet 10M is determined to be normal, the controller 26 assesses whether the loading state abnormality check has been performed a predetermined number of times (step S103). The predetermined number of times is either one or a multiple. If it is determined that the loading state abnormality check has been performed a predetermined number of times, the controller 26 terminates this process. If it is determined that the loading state abnormality check has not been performed a predetermined number of times, the controller 26 repeats the preceding processing in step S101.

[0041] If in step S102 it is determined that the current loading state of the loaded pallet 10M is abnormal, the controller 26 assesses, based on the modeled data stored in the memory unit 24, whether a load displacement can be avoided (step S104).

[0042] If it is determined that a cargo shift can be avoided, the controller 26 issues a directive to execute the contingency plan (step S105) and carries out the process in step S103. If it is determined that a cargo shift cannot be avoided, the controller 26 issues a directive to stop the cargo handling operation (step S106) and terminates this process.

[0043] Here, the load state information reference unit 31 performs the processing in steps S101 and S103. The load state abnormality determination unit 32 performs the processing in step S102. The avoidance possibility determination unit 33 executes the process from step S104 to step S106.

[0044] Fig. Figure 5 is a flowchart that specifies the process for determining the work situation, which is to be carried out by the controller 26. This process is carried out by the work situation information acquisition unit 34, the work situation abnormality determination unit 35, and the avoidance possibility determination unit 36. This process is executed when the load handling work begins.

[0045] In Fig. Step 5: The controller 26 first receives the image data from the camera 21, the sound data from the microphone 22, and the vibration data from the accelerometer 23 (step S111). Subsequently, based on the image data from the camera 21, the sound data from the microphone 22, and the vibration data from the accelerometer 23, the controller 26 assesses whether the current working situation is normal (step S112).

[0046] In a case where the load M is being towed, the towing process appears in the captured image, and noises and vibrations attributable to the towing occur. Therefore, a comprehensive assessment can be made from the image, noise, and vibration data to determine whether an abnormal situation is occurring that is causing the load to shift. For example, even if the towing of load M is not detected by the image data due to a blind spot of camera 21, it can still be detected by the noise and vibration data. In this case, a threshold for assessing the abnormality of the work situation is also calculated based on the modeled data stored in memory unit 24.

[0047] If the current work situation is determined to be normal, the controller 26 assesses whether the load handling work is complete (step S113). If it is determined that the load handling work is complete, the controller 26 terminates this process. If it is determined that the load handling work is not complete, the controller 26 repeats the preceding process in step S111.

[0048] If in step S112 it is determined that the current work situation is abnormal, the controller 26 uses the modeled data stored in the memory unit 24 to assess whether a load displacement can be avoided (step S114).

[0049] If it is determined that a cargo shift is avoidable, the controller 26 issues a directive to execute the contingency plan (step S115) and carries out the process in step S113. If it is determined that the cargo shift is unavoidable, the controller 26 issues a directive to stop the cargo handling operation (step S116) and terminates this process.

[0050] Here, the work situation information acquisition unit 34 performs the processing in steps S111 and S113. The work situation abnormality determination unit 35 performs the process in step S112. The avoidance possibility determination unit 36 ​​performs the process from step S114 to step S116.

[0051] Fig. Figure 6 is a flowchart that specifies the process of contingency plan generation to be performed by the contingency plan generation unit 37 of the controller 26. This processing is executed when the processing to determine the load state and the processing to determine the work situation are performed.

[0052] In Fig. 6. The controller 26 first assesses whether a contingency plan should be executed, based on the processing described above in steps S105 and S115 (step S121). If it decides to execute the contingency plan, the controller 26 obtains the image data from camera 21, the sound detection data from microphone 22, and the vibration data from accelerometer 23 (step S122).

[0053] Subsequently, the controller 26 generates evasive maneuver data to prevent cargo shifts using the image data from camera 21, the noise detection data from microphone 22, the vibration data from accelerometer 23, and the modeled data stored in memory unit 24 (step S123). Specifically, the controller 26 receives operating data corresponding to the image data from camera 21, the noise detection data from microphone 22, and the vibration data from accelerometer 23 from the modeled data, and generates control command data for the drive unit 2 and the cargo handling unit 3, which correspond to the operating data as the evasive maneuver data.

[0054] As in Fig. As illustrated in Figure 3, for example, in a case where the load M on the second stage, loaded on pallet 10 on the right, collides with the load M loaded on pallet 10 on the left (see A in the figure), avoidance plan data is generated so that the right-loaded pallet 10M is picked up before the left-loaded pallet 10M, even though pallets are normally picked up from left to right. Furthermore, if it is difficult to pick up the right-loaded pallet 10M before the left-loaded pallet 10M, avoidance plan data is generated in which the left-loaded pallet 10M is picked up while being shifted to the left.

[0055] Fig. Figure 7 is a flowchart that indicates the processing process of the load handling control unit, which is to be executed by the load handling control unit 38 of the controller 26. This processing is carried out when the processing to determine the loading status and the processing to determine the working situation are performed.

[0056] In Fig. 7. The controller 26 first assesses whether the load handling operation should be stopped or not, based on the processing described above in steps S106 and S116 (step S131). If it is decided that the load handling operation should not be stopped, the controller 26 assesses, based on the processing described above in steps S105 and S115 (step S132), whether the alternative plan should be executed or not.

[0057] If it is decided that the alternative plan should not be executed, the control unit 26 controls the drive unit 2 and the load handling unit 3 so that the load handling work is carried out as usual (step S133). Normal load handling work includes, for example, picking up pallets in a predetermined sequence.

[0058] If the decision is made to execute the alternative plan, the controller 26 controls the drive unit 2 and the load handling unit 3 to carry out the load handling work in accordance with the alternative plan data generated during the preceding processing in step S123 (step S134).

[0059] If, in step S131, the decision is made to stop the load handling operation, the controller 26 controls the drive unit 2 and the load handling unit 3 to forcibly stop the execution of the load handling operation (step S135). Subsequently, the controller 26 controls the alarm 25 to issue an alarm (step S136).

[0060] As described above, in the present embodiment, before the start of the load handling operation, the load M on the pallet 10 held by the forks 11 is detected as a loading condition together with the pallet 10 using the camera 21, and it is determined whether the loading condition is abnormal. If it is determined that the loading condition is abnormal, it is possible to suppress the occurrence of load displacements due to interference or similar factors between adjacent loads M by stopping the load handling operation or by performing an operation to prevent load displacements. After the start of the load handling operation, the working situation is detected using the camera 21, the microphone 22, and the accelerometer 23, and it is determined whether the working situation is abnormal.If the work situation is determined to be abnormal, it is possible to suppress the occurrence of load shifts caused by a situation that leads to load shifts, such as the pulling of load M, by stopping the load handling work or by performing an operation to prevent load shifts. As described above, the occurrence of load shifts can be suppressed by continuously monitoring the work situation during load handling work, even if an abnormality in the load condition is missed during the initial load condition assessment.

[0061] Furthermore, in the present embodiment, it is possible to easily and cost-effectively record the loading status, including the cargo M and the pallet 10, using the camera 21.

[0062] In the present embodiment, it is possible to detect an abnormality in the working situation over a wide area and with high accuracy using the camera 21, the microphone 22 and the accelerometer 23, which leads to a displacement of the load, such as the dragging of the load M and the collision between the loads M that occur during the load handling work.

[0063] Furthermore, in the present embodiment, using the modeled data, in which the operating data of the forklift 1 are linked by the operator with the image data captured by the camera 21, the noise data captured by the microphone 22, and the vibration data of the accelerometer 23 at the time of operation of the forklift 1, it is determined whether the load condition and the working situation are abnormal, based on a result of the modeling of the actual abnormality determination and assessment by the operator. It is therefore possible to determine in a suitable manner whether the load condition and the working situation are abnormal or not.

[0064] Furthermore, in the present embodiment, if it is determined that the loading condition or the working situation is abnormal, evasive plan data is generated to prevent load shifting, and the forklift 1 is controlled to perform the load handling work according to the evasive plan data. In this way, the occurrence of load shifting is automatically suppressed.

[0065] Furthermore, in the present embodiment, if the loading condition or work situation is determined to be abnormal, the contingency plan data is generated when it is determined that the load displacement is avoidable, and the forklift 1 is controlled to stop the load handling operation when it is determined that the load displacement is unavoidable. The contingency plan data is thus generated as needed, eliminating the need to generate it unnecessarily. This simplifies processing in the controller 26.

[0066] Furthermore, in the present embodiment, using the modeled data, in which the operating data of the forklift 1 are linked by the operator with the image data of the camera 21, the noise detection data of the microphone 22, and the vibration data of the accelerometer 23 recorded at the time of operation of the forklift 1, it is determined whether the load displacement of the load M is avoidable, based on a result of the modeling of the actual abnormality detection and assessment by the operator. Thus, it can be appropriately determined whether a load displacement of the load M is avoidable or not.

[0067] Furthermore, in the present embodiment, the evasive maneuver data is generated using the modeled data, in which the operating data of the forklift 1 are linked by the operator with the image data from camera 21, the noise recording data from microphone 22, and the vibration data from accelerometer 23, which were recorded at the time the forklift 1 was in operation. This makes it possible to obtain the evasive maneuver data based on the result of modeling the actual abnormality detection and assessment by the operator. In this way, the occurrence of load shifts can be suppressed appropriately.

[0068] It should be noted that the present disclosure is not limited to the preceding embodiment. In the present embodiment, for example, the loading status of the loaded pallet 10M is detected by the camera 21 before the start of the load handling work, but the present disclosure is not limited to such a form. For example, the loading status of the loaded pallet 10M can be detected with a laser sensor such as LiDAR instead of the camera 21 or together with the camera 21. In other words, it is only necessary to detect the loading status of the loaded pallet 10M with at least one type of sensor.

[0069] In the present embodiment, the work situation after the start of the load handling work is detected using the camera 21, the microphone 22 and the accelerometer 23, but the present disclosure is not particularly limited to this, and the work situation can be detected with a variety of sensor types.

[0070] In the preceding embodiment, the storage unit 24 stores the modeled data in which the operating data of the forklift 1, entered by the operator, is linked with the image data from the camera 21, the sound recording data from the microphone 22, and the vibration data from the accelerometer 23, which were recorded at the time the forklift 1 was operating. However, the present disclosure is not limited to this form. The storage unit 24 can store modeled data in which the operating data of the forklift 1, entered by the operator, is linked with the image data from the camera 21, the sound recording data from the microphone 22, and the vibration data from the accelerometer 23, which were recorded at the time the forklift 1 was operating. The storage unit 24 can also store modeled data in which the operating data of the forklift 1, entered by the operator, is linked with information about the load status and the working situation.

[0071] Furthermore, in the preceding embodiment, the modeled data obtained by modeling the identification, assessment, and operation of the person with respect to various loading states and work situations are used to determine the loading state and work situation of the loaded pallet 10M, to determine whether a load shift is avoidable, and to generate the data for the contingency plan. However, the present disclosure is not specifically limited to such a form, and data or similar information obtained by establishing rules in advance may be used instead of data obtained by modeling the identification, assessment, and operation of the person.

[0072] In the above embodiment, the load receiving work is carried out as load handling work, but the load handling work is not specifically limited to the load receiving work, and, for example, a so-called unloading work can be carried out in which the loaded pallet 10M is placed at a position next to another already existing loaded pallet 10M.

[0073] In the preceding embodiment, the load handling work is performed by automatic operation of the forklift 1. However, the load handling system of this disclosure can be used to support the load handling work by manual operation of the forklift 1 or by remote control of the forklift 1. In this case, for example, notification and alarm of the detection result of the loading status of the loaded pallet 10M and the work situation, instructions to prevent load shifting, instructions to stop work, and the like are performed.

[0074] In the present embodiment, the sensor, such as the camera 21 for detecting the loading status of the loaded pallet 10M and the working situation, is mounted on the forklift 1, but the present disclosure is not specifically limited to this, and the sensor can be installed on the infrastructure side. In this case, output data from the sensor can be transmitted to the forklift 1 via wireless communication or similar means, or the processing by the controller 26 can be performed on the infrastructure side. Reference symbol list 1 forklift 10 pallets 11 Fork 20 Cargo handling system 21 Camera (sensor, first detection unit, second detection unit) 22 Microphone (sensor, second detection unit) 23 Accelerometers (vibration sensor, sensor, second detection unit) 24 storage units 31 Load status information reference unit (first acquisition unit) 32 Load condition abnormality determination unit (first abnormality determination unit) 33 Avoidance Possibility Determination Unit 34 Work situation information recording unit (second recording unit) 35 Work situation abnormality determination unit (second abnormality determination unit) 36 Avoidance Possibility Determination Unit 37 Backup Plan Generation Unit 38 Load handling control unit M cargo

Claims

[1] Load handling system (20) that performs load handling operations with a forklift truck, wherein the load handling system comprises: a first detection unit (21, 31) configured to detect, prior to the commencement of load handling work, a load on a pallet held by a fork of the forklift, together with the pallet as a load state using at least one type of sensor; a first abnormality detection unit (32) configured to determine whether the loading state detected by the first detection unit is abnormal; a second detection unit (34) configured to detect a work situation using a variety of sensor types after the load handling work has started; a second abnormality detection unit (35) configured to determine whether the work situation detected by the second detection unit is abnormal; a storage unit (24) configured to store modeled data in which forklift operating data is linked by an operator with information about the load state captured by the first acquisition unit and information about the work situation captured by the second acquisition unit when the load handling work is performed by manual operation of the forklift, wherein The first abnormality detection unit (32) determines whether the loading state detected by the first detection unit is abnormal, by using the modeled data stored in the storage unit, and The second abnormality detection unit (35) determines whether the work situation detected by the second detection unit is abnormal by using the modeled data stored in the storage unit. [2] Cargo handling system (20) according to claim 1, wherein one type of sensor is a camera that captures an image of the pallet and the cargo. [3] Load handling system (20) according to claim 1, wherein the plurality of sensor types comprises a camera that captures an image of the pallet and the load, a microphone that detects sounds in the vicinity of the forklift, and a vibration sensor that measures vibrations acting on the forks. [4] Cargo handling system (20) according to claim 1, wherein the modeled data are obtained by collecting pre-existing data relating to recognition, assessment and operation by a human with regard to various loaded states and work situations, and by performing a modeling by deep learning using the collected data. [5] Cargo handling system (20) according to claim 1, further comprising: a contingency plan generation unit (37) configured to generate contingency plan data to avoid cargo displacement when the first abnormality determination unit determines that the loading condition is abnormal, or when the second abnormality determination unit determines that the work situation is abnormal; and a load handling control unit (38) configured to control the forklift truck to perform the load handling work according to the avoidance plan data generated by the avoidance plan generation unit. [6] Cargo handling system (20) according to claim 5, further comprising: an avoidance possibility determination unit (33, 36) configured to determine whether a load displacement is avoidable when the first load state abnormality determination unit determines that the load state is abnormal, or when the second abnormality determination unit determines that the work situation is abnormal, wherein the alternative plan generation unit (37) generates the alternative plan data when the avoidance possibility determination unit determines that the cargo displacement is avoidable, and the load handling control unit (38) controls the forklift to stop the load handling operation when the avoidance possibility determination unit determines that the load displacement is unavoidable. [7] Cargo handling system (20) according to claim 6, wherein the avoidance possibility determination unit (33, 36) determines, based on the modeled data stored in the storage unit, whether the cargo displacement is avoidable. [8] Cargo handling system (20) according to claim 5, wherein the avoidance plan generation unit (37) generates the avoidance plan data using the information about the loading state detected by the first detection unit and the modeled data stored in the storage unit in a case where the first abnormality determination unit determines that the loading state is abnormal and when it is determined that the cargo displacement is avoidable, and generates the avoidance plan data using the information about the working situation detected by the second detection unit and the modeled data stored in the storage unit in a case where the second abnormality determination unit determines that the working situation is abnormal and when it is determined that the cargo displacement is avoidable.

Images