Fork position adjustment device

Abstract

To provide a fork position adjustment device capable of improving work efficiency of a forklift by suppressing a time required for positioning a fork for an insertion port.SOLUTION: A fork position adjustment device applied to a forklift comprises: an acquisition unit that acquires image data; a storage device that stores a learned model for outputting an index indicative of a height of an insertion port; and a control unit that acquires the height of the insertion port on the basis of output results of the learned model and controls a lifting device. The control unit executes estimation processing S3 for estimating a pallet having a target port into which a fork is inserted as a result of operation of a lift lever in an operated state, and lifting assistance processing S5 for controlling the lifting device such that the fork faces the target port when a difference ΔH between the height of the target port and the actual height of the fork as a result of the operation of the lift lever becomes a predetermined value ΔW1 indicative of the vicinity of the target port.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a fork position adjustment device. [Background technology]

[0002] Patent Document 1 describes a loading / unloading operation support device mounted on a forklift. When using the forklift to unload a load, the operator aligns the forks so that they face the fork insertion end face of the pallet. The fork insertion end face has an insertion hole into which the fork is inserted.

[0003] The loading and unloading operation support device includes a camera and a display device. The display device includes a display screen. An image of the front of the forks captured by the camera is displayed on the display screen. The fork insertion end face and the insertion port are displayed on the display screen. A cross-sectional image of the forks is displayed on a reduced scale on the display screen. The passenger aligns the forks so that the cross-sectional image of the forks fits within the insertion port on the display screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-8367 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the alignment of the forks with respect to the insertion opening depends on the operator's operating technique. Depending on the operator's operating technique, it may take time to align the forks with the insertion opening, which may result in the forklift being stopped for a long time to perform the loading operation. This may result in a decrease in the work efficiency of the forklift. [Means for solving the problem]

[0006] The fork position adjustment device that solves the above problem is a fork position adjustment device that is applied to a forklift equipped with a fork, a lifting device that raises and lowers the fork, and a lift lever that is operated by a rider to operate the lifting device, and includes: an acquisition unit that acquires image data including an insertion port and the pallet from a camera, where the hole into which the fork is inserted to load a pallet on the fork is defined as an insertion port; a storage device that stores a trained model that outputs the height of the insertion port included in the image data or an index indicating the height of the insertion port when the image data is input; and a pre-training unit that outputs a pre-training unit based on an output result of the trained model. and a control unit that acquires the height of the insertion opening contained in the image data and controls the lifting device, wherein the control unit executes an estimation process that estimates the pallet having a target opening, which is the insertion opening into which the forks are to be inserted, from at least one of the operation of the lift lever in a state in which it is being operated and the operation history of the lift lever, and a lifting assistance process that controls the lifting device so that the forks face the target opening when the difference between the acquired height of the target opening and the actual height of the forks as a result of the operation of the lift lever becomes equal to or less than a predetermined value indicating the proximity of the target opening while the lift lever is being operated by a passenger.

[0007] According to the above configuration, the estimation process estimates a pallet having a target opening for inserting the forks. Then, the occupant operates the lift lever to raise and lower the forks to the vicinity of the target opening, and the control unit executes the lift assist process so that the forks face the target opening. In other words, the lift assist process assists the occupant in aligning the forks. Therefore, the work efficiency of the forklift can be improved by reducing the time required to align the forks with the insertion opening.

[0008] In the fork position adjusting device described above, the control unit may execute the estimation process based on the operation of the lift lever in the operated state. According to the above configuration, an estimation process that directly reflects the intention of the passenger is executed, so that the estimation accuracy of the estimation process can be improved.

[0009] In the fork position adjusting device described above, the control unit may execute the estimation process from an operation history of the lift lever. According to the above configuration, executing the estimation process from the lift lever operation history means executing the estimation process from many lift lever operation modes, which improves the estimation accuracy of the estimation process, and therefore allows the lift assist process to be executed appropriately. Effect of the Invention

[0010] According to the present invention, the time required for aligning the forks with the insertion openings can be reduced, thereby improving the work efficiency of the forklift. [Brief description of the drawings]

[0011] [Figure 1] FIG. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a first embodiment of a fork position adjusting device. [Diagram 3] 4 is a flowchart of a process executed by a control device of the first embodiment. [Figure 4] FIG. 13 is a diagram showing a state before the forks are aligned. [Diagram 5] 5 is a diagram showing imaging data obtained when an image is captured at the position indicated by the arrow 5 in FIG. 4. FIG. [Figure 6] 5 is a diagram showing imaging data obtained when an image is captured at the position indicated by an arrow 6 in FIG. 4. FIG. [Figure 7] 5 is a diagram showing imaging data obtained when an image is captured at the position indicated by the arrow 7 in FIG. 4. FIG. [Figure 8] 4A to 4C are diagrams illustrating an operation state of a lift lever. [Figure 9] FIG. 13 is a diagram showing a state in which the forks are raised. [Figure 10] FIG. 11 is a diagram showing a storage unit in a second embodiment of the fork position adjusting device. [Figure 11] FIG. 11 is a diagram comparing the operation state of the lift lever with the operation history of the lift lever. [Figure 12] FIG. 11 is a diagram comparing the lifting state of the forks with the lifting history of the forks. [Figure 13] 10 is a flowchart of a process executed by a control device in a modified example of the fork position adjusting device. [Figure 14] 13A and 13B are diagrams illustrating tilt processing in a modified example. [Figure 15] 13A to 13C are diagrams illustrating a correction process in a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] [First embodiment] A first embodiment of a fork position adjusting device will now be described with reference to Figures 1 to 9. The fork position adjusting device of this embodiment is mounted on a forklift, so the configuration of the forklift will be described first.

[0013] <Forklift configuration> 1, the forklift 10 includes a lift lever 17, a tilt lever 18, and a cargo handling device 11. The forklift 10 of this embodiment is operated by a person riding on the forklift 10.

[0014] The lift lever 17 and the tilt lever 18 can be tilted forward or backward from a neutral position by the rider of the forklift 10. When the lift lever 17 is tilted forward, the forks 16 descend, and when the lift lever 17 is tilted backward, the forks 16 rise. When the tilt lever 18 is tilted forward, the forks 16 tilt forward, and when the tilt lever 18 is tilted backward, the forks 16 tilt backward. Tilting the lift lever 17 and the tilt lever 18 forward or backward is referred to as operating the lift lever 17 and the tilt lever 18. Incidentally, when the forks 16 tilt forward, the forks 16 tilt so that their tips point downward. When the forks 16 tilt backward, the forks 16 tilt so that their tips point upward.

[0015] The loading device 11 includes a mast 12, a lift bracket 15, a fork 16, a lift cylinder 20, and a tilt cylinder 21. The mast 12 includes an outer mast 13 and an inner mast 14. The outer mast 13 is supported by the vehicle body so as to be tiltable back and forth.

[0016] The inner mast 14 is provided so as to be capable of ascending and descending relative to the outer mast 13. The lift bracket 15 is fixed to the inner mast 14. The fork 16 is fixed to the lift bracket 15. The lift bracket 15 and the fork 16 ascend and descend together with the inner mast 14.

[0017] The lift cylinder 20 and the tilt cylinder 21 are hydraulic cylinders. The lift cylinder 20 raises and lowers the inner mast 14 by supplying and discharging hydraulic oil through operation of the lift lever 17. The tilt cylinder 21 tilts the mast 12 by supplying and discharging hydraulic oil through operation of the tilt lever 18.

[0018] As shown in FIG. 2, the forklift 10 includes a hydraulic mechanism 22 and a fork position adjustment device 30. The hydraulic mechanism 22 is a member for making the forklift 10 perform loading and unloading. The hydraulic mechanism 22 includes a loading motor for driving a pump and a control valve for controlling the supply and discharge of hydraulic oil to the lift cylinder 20 and the tilt cylinder 21. The lift lever 17 and the tilt lever 18 are mechanically connected to the control valve. The control valve can be switched between an open and closed state by operating the lift lever 17 and the tilt lever 18. The control valve also functions as an electromagnetic control valve for adjusting the opening degree of an oil passage for supplying and discharging hydraulic oil to the lift cylinder 20 and the tilt cylinder 21. The lift cylinder 20 and the hydraulic mechanism 22 are a lifting device for lifting and lowering the fork 16. The lift lever 17 is a lever operated by a rider to operate the lifting device. The tilt cylinder 21 and the hydraulic mechanism 22 are a tilting device for tilting the fork 16. The tilt lever 18 is a lever operated by a passenger to operate the tilt device.

[0019] <Configuration of fork position adjustment device> The fork position adjustment device 30 includes a lift potentiometer 17 a, a tilt potentiometer 18 a, a camera 31, a control device 41, and an auxiliary storage device 51.

[0020] The lift potentiosensor 17a outputs a signal Sa corresponding to the tilt angle Lθ of the lift lever 17. The tilt potentiosensor 18a outputs a signal Sb corresponding to the tilt angle of the tilt lever 18.

[0021] The camera 31 is a digital camera. The camera 31 includes an imaging element. Examples of the imaging element include a charge coupled device (CCD) image sensor and a complementary metal oxide semiconductor (CMOS) image sensor. Examples of the camera 31 include an RGB camera, an infrared camera, a grayscale camera, and a visible light camera. The camera 31 captures images at a predetermined frame rate to generate image data. This image data is digital data of an image captured by the camera 31.

[0022] As shown in FIG. 4, the pallet P to be loaded on the fork 16 has an insertion port P1. The insertion port P1 is a hole into which the fork 16 is inserted in order to load the pallet P on the fork 16. The camera 31 is provided on the upper part of the mast 12. The camera 31 is disposed on the mast 12 so as to capture an image of the pallet P to be loaded on the fork 16 and its insertion port P1 before the pallet P is loaded on the fork 16. Specifically, the camera 31 is disposed on the mast 12 so as to capture an image of the pallet P placed on the loading platform 100 and its insertion port P1. In this embodiment, the timing at which the camera 31 captures the image of the pallet P and its insertion port P1 is before the pallet P is loaded on the fork 16 and when the forklift 10 stops in front of the loading platform 100 to load the pallet P.

[0023] As shown in FIG. 2, the control device 41 includes a processor 42 and a storage unit 43. The processor 42 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), and a digital signal processor (DSP). The storage unit 43 includes a random access memory (RAM) and a read only memory (ROM). The storage unit 43 stores program codes or instructions configured to cause the processor 42 to execute processes. The storage unit 43, i.e., the computer readable medium, includes any available medium accessible by a general-purpose or dedicated computer. The control device 41 may be configured by a hardware circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control device 41, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as an ASIC or an FPGA, or a combination thereof.

[0024] The control device 41 is electrically connected to each of the lift potentiometer 17a, the tilt potentiometer 18a, and the camera 31. Signals Sa and Sb are input to the control device 41. The control device 41 is electrically connected to each of the control valves and the loading motors of the hydraulic mechanism 22. The control device 41 can control the hydraulic mechanism 22. The control device 41 can control the supply and discharge of hydraulic oil to the lift cylinder 20 and the tilt cylinder 21 by controlling the control valves and the loading motors of the hydraulic mechanism 22, regardless of the operation of the lift lever 17 and the tilt lever 18. The control device 41 can control the lifting device and the tilting device.

[0025] The control device 41 includes an acquisition unit 44 and a control unit 45. The acquisition unit 44 and the control unit 45 are functional elements that function as a result of the control device 41 executing a program. The acquisition unit 44 acquires image data including the socket P1 and the palette P from the camera 31.

[0026] The auxiliary storage device 51 stores information that can be read by the control device 41. Examples of the auxiliary storage device 51 include a hard disk drive and a solid state drive.

[0027] The auxiliary storage device 51 is a storage device that stores the trained model M1. When image data is input, the trained model M1 outputs an index indicating the height Ho of the socket included in the image data.

[0028] When image data is input, the trained model M1 of this embodiment outputs an indicator of whether the input image data corresponds to a predetermined class. A plurality of classes are set according to the height Ho of the socket P1, such as "the socket P1 is located at a height H0", "the socket P1 is located at a height H1", "the socket P1 is located at a height H2", etc.

[0029] Probability is used as an index indicating whether image data corresponds to each class. Probability can also be said to be the reliability that image data corresponds to a class. As the probability, for example, class probability and score can be used. Class probability is the probability that image data corresponds to a class. Class probability is a value between "0" and "1". In this embodiment, class probability is used as an index indicating whether image data corresponds to each class. This class probability is an index indicating the height Ho of the socket P1.

[0030] The trained model M1 is generated by machine learning using a DNN (Deep Neural Network). For example, a CNN (Convolution Neural Network) can be used as a machine learning algorithm. The CNN determines the corresponding class for each image data. The CNN includes an input layer, an intermediate layer, and an output layer. Image data is input to the input layer. The intermediate layer includes a convolution layer and a pooling layer. The intermediate layer extracts features of the image data. The output layer outputs class probabilities.

[0031] The trained model M1 is generated by supervised learning or semi-supervised learning using training data. The training data can be labeled image data.

[0032] 4 and 5, for example, image data showing a pallet P with its insertion port P1 at a height H0 is labeled with "insertion port P1 is at a height H0" and used as training data. The image data shown in FIG. 5 was captured when the forklift 10 stopped at a predetermined position T1 in front of the platform 100 before loading the pallet P onto the forks 16.

[0033] 4 and 6, for example, image data showing a pallet P with its insertion port P1 at a height H1 is labeled with "insertion port P1 is at a height H1" and used as training data. The image data shown in FIG. 6 was captured when the forklift 10 stopped at a predetermined position T1 in front of the platform 100 before loading the pallet P onto the forks 16.

[0034] 4 and 7, for example, image data showing a pallet P with its insertion port P1 at a height H2 is labeled with "insertion port P1 is at a height H2" and used as training data. The image data shown in FIG. 7 was captured when the forklift 10 stopped at a predetermined position T1 in front of the platform 100 before loading the pallet P onto the forks 16.

[0035] <Aligning the fork with the insertion hole> The control device 41 executes a process of aligning the forks 16 with respect to the insertion port P1 before loading the pallet P onto the forks 16. The process of aligning the forks 16 with respect to the insertion port P1 starts when the forklift 10 stops in front of the platform 100 and the driver starts to tilt the lift lever 17 backward.

[0036] 3, the control device 41 first executes the process of step S1. In the process of step S1, the control device 41 acquires image data from the camera 31. The process of step S1 is executed by the acquisition unit 44 of the control device 41. After executing the process of step S1, the control device 41 executes the process of step S2.

[0037] In the process of step S2, the control device 41 acquires the height Ho of the socket P1 included in the image data. The process of step S2 is executed by the control unit 45. The control unit 45 inputs the image data acquired by the acquisition unit 44 to the trained model M1 of the auxiliary storage device 51. When the image data is input, the trained model M1 outputs the class probability of the class corresponding to the height Ho of the socket P1 included in the input image data. In addition, when the image data includes multiple pallets P and multiple sockets P1, the trained model M1 outputs the class probability of the class corresponding to the height Ho of each socket P1. The control unit 45 acquires the class probability output from the trained model M1. The control unit 45 acquires the height Ho of the socket P1 based on the acquired class probability. As a result, the control unit 45 acquires the height Ho of the socket P1 of the pallet P placed on the placement table 100. In other words, the control unit 45 acquires the height Ho of the socket P1 included in the image data based on the output result of the trained model M1. After executing the process of step S2, the control device 41 advances the process to step S3.

[0038] The control device 41 executes an estimation process in the process of step S3. Hereinafter, step S3 will be referred to as estimation process S3. The estimation process S3 is executed by the control unit 45. The estimation process S3 is a process for estimating a pallet P including a target opening Pg, which is a target insertion opening P1 into which the forks 16 are inserted, based on the operation of the lift lever 17 in an operated state.

[0039] 8, in estimation process S3, the control unit 45 acquires a signal Sa. The control unit 45 acquires a tilt angle Lθ of the lift lever 17 based on the signal Sa. The control unit 45 monitors the operation of the lift lever 17 by monitoring a change in the tilt angle Lθ with respect to a time t of the operation of the lift lever 17.

[0040] The control unit 45 estimates the pallet P including the target opening Pg when the tilt angle Lθ continues to be within a predetermined range ΔLθ until a predetermined time t1, the tilt angle Lθ starts to decrease, and the decreasing trend of the tilt angle Lθ continues for a predetermined time Δt. The predetermined range ΔLθ indicates the range of the tilt angle Lθ within which the rider can operate the lift lever 17 to lift the forks 16 toward the target opening Pg. The tilt angle Lθ of the lift lever 17 starts to decrease, which means that the rider has started to adjust the position of the forks 16 with respect to the target opening Pg.

[0041] As shown in FIG. 9, in the estimation process S3, the control unit 45 calculates the amount of change ΔHL in the height HL of the fork 16 from time t1 to time t1′ when a predetermined time Δt has elapsed. When the lift lever 17 is operated so that the change amount ΔHL continues after time t1', the control unit 45 estimates the height Ho of the insertion port P1 acquired in the processing of step S2. The estimated height Ho is, for example, the height that is realized earliest among the heights H0, H1, and H2. The control unit 45 estimates the insertion port P1 located at the height that is realized earliest after time t1 as the target port Pg. The control unit 45 estimates the target port Pg from the change amount ΔHL in a predetermined time Δt. In other words, the control unit 45 estimates the pallet P having the target port Pg into which the fork 16 is inserted from the operation amount of the lift lever 17 in the predetermined time Δt. In this embodiment, the control unit 45 sets the insertion port P1 located at the height H0 as the target port Pg.

[0042] As shown in FIG. 3 and FIG. 9, when the control device 41 executes the estimation process S3, the control device 41 advances the process to step S4. When the control device 41 executes the process of step S4, the rider continues to operate the lift lever 17, and the forks 16 continue to rise. In the process of step S4, the control device 41 determines whether the difference ΔH between the height Hog of the target opening Pg and the actual height HL of the forks 16 is a predetermined value ΔW1. As the forks 16 rise, the difference ΔH decreases. The height HL of the actual forks 16 is calculated by the control unit 45 based on the operation of the lift lever 17. The predetermined value ΔW1 is set to, for example, the thickness of the pallet P. The predetermined value ΔW1 is a value indicating the vicinity of the target opening Pg. The predetermined value ΔW1 may be calculated by the control device 41 based on, for example, image data captured by the camera 31.

[0043] In this case, the trained model M1 learns data in which image data depicting each of a plurality of types of pallets P is labeled with the thickness of each pallet P as training data. Then, the control device 41 may calculate the thickness of the pallet P calculated based on the index of the thickness of the pallet P output from the trained model M1 as a predetermined value ΔW1. Furthermore, the data output from the trained model M1 may be the thickness of the pallet P itself.

[0044] When the control device 41 determines that the difference ΔH has not reached the predetermined value ΔW1 while the lift lever 17 is being operated (step S4: NO), it continues with step S4. When the control device 41 determines that the difference ΔH has reached the predetermined value ΔW1 while the lift lever 17 is being operated (step S4: YES), it advances the process to step S5. In other words, when the difference ΔH between the height Hog of the target opening Pg and the actual height HL of the fork 16 due to the operation of the lift lever 17 reaches the predetermined value ΔW1 while the lift lever 17 is being operated by the rider, the control device 41 advances the process to step S5.

[0045] The control device 41 executes the lift assist process S5 in the process of step S5. Hereinafter, step S5 will be referred to as the lift assist process S5. The lift assist process S5 is executed by the control unit 45. The lift assist process S5 is a process in which the control unit 45 controls the lift device so that the fork 16 faces the target opening Pg. The time when the lift assist process S5 starts is set to time t2. The lift assist process S5 is a process in which the control unit 45 controls the hydraulic mechanism 22 to lift the fork 16 even if the passenger continues to operate the lift lever 17 after time t2. That is, in the lift assist process S5, the operation of the lift lever 17 by the passenger is not reflected in the lift of the fork 16. In the lift assist process S5, the control unit 45 supplies hydraulic oil to the lift cylinder 20 by controlling the hydraulic mechanism 22 so that the fork 16 rises by a predetermined value ΔW1. As a result, the fork 16 faces the target opening Pg.

[0046] After executing the lift assist process S5, the control device 41 ends the process of aligning the fork 16 with the insertion port P1. <Action of this embodiment> The operation of this embodiment will now be described.

[0047] The estimation process S3 estimates the pallet P having the target opening Pg into which the forks 16 are inserted. Then, the passenger operates the lift lever 17 to raise and lower the forks 16 to the vicinity of the target opening Pg, and then the control unit 45 executes the lift assistance process S5 so that the forks 16 face the target opening Pg. In other words, the lift assistance process S5 assists the passenger in aligning the forks 16.

[0048] <Effects of this embodiment> The effects of this embodiment will be described. (1-1) By executing the estimation process S3 and the lift assistance process S5, the positioning of the forks 16 with respect to the target opening Pg of the rider is assisted. Therefore, the time required for aligning the forks 16 with respect to the insertion opening P1 is reduced, thereby improving the work efficiency of the forklift 10.

[0049] (1-2) The control device 41 executes the estimation process S3 based on the operation of the lift lever 17. Since the estimation process S3 that directly reflects the passenger's intention is executed, the estimation accuracy of the estimation process S3 can be improved. Therefore, the lift assistance process S5 can be executed appropriately.

[0050] (1-3) In recent years, the occupants of the forklift 10 have been aging. As the occupants age, it tends to take longer to align the forks 16 with the insertion port P1. In this regard, according to the present embodiment, the occupant is assisted in aligning the forks 16 with the target port Pg, so that the deterioration of workability due to the aging of the occupant can be suppressed.

[0051] [Second embodiment] A second embodiment of a fork position adjusting device will be described below with reference to Figs. 10 to 12. The main difference from the first embodiment is that the method of the estimation process S3 is different. The main differences from the first embodiment will be described in detail below, and the same components as in the first embodiment will be denoted by the same reference numerals and detailed description will be omitted.

[0052] <Storage section> 10, the memory unit 43 stores a history of past alignment processes of the fork 16 with respect to the insertion port P1. For example, the memory unit 43 stores a plurality of data including data D1, D2, D3, and D4 that link the operation of the lift lever 17 performed by the rider in the past with the change in height of the fork 16 corresponding to the operation of the lift lever 17. The plurality of data including data D1, D2, D3, and D4 can be said to be the operation history of the lift lever 17.

[0053] <Estimation process> The estimation process S3 of this embodiment will be described with reference to Figures 11 and 12. The two-dot chain lines in Figures 11 and 12 indicate data D1 stored in the storage unit 43. The one-dot chain lines in Figures 11 and 12 indicate data D2 stored in the storage unit 43. The solid lines in Figures 11 and 12 indicate the actual operation of the lift lever 17 and the movement of the forks 16 in this embodiment.

[0054] As shown in Figures 11 and 12, in the estimation process S3 of this embodiment, the control unit 45 refers to data D1, D2 linked to the operation of the lift lever 17 similar to the operation of the lift lever 17 from time t1 to the time a predetermined time Δt has elapsed.

[0055] Among a plurality of data including data D1, D2, D3, and D4, the data which tilt the lift lever 17 more than the current operation of the lift lever 17 after time t1 are compared with each other. As a result of the comparison, the data which is closest to the current operation of the lift lever 17 after time t1 is data D1.

[0056] Among a plurality of pieces of data including data D1, D2, D3, and D4, the data in which the change in the height HL of the fork 16 is greater than the change in the current height HL of the fork 16 after time t1 is compared with each other. As a result of this comparison, the data that is closest to the change in the height HL of the fork 16 after time t1 is data D1.

[0057] Among a plurality of data including data D1, D2, D3, and D4, the data that tilt the lift lever 17 less than the current operation of the lift lever 17 after time t1 are compared with each other. As a result of the comparison, the data that is closest to the current operation of the lift lever 17 after time t1 is data D2.

[0058] Among a plurality of pieces of data including data D1, D2, D3, and D4, the data in which the change in the height HL of the fork 16 is smaller than the change in the current height HL of the fork 16 after time t1 is compared with each other. As a result of this comparison, the data that is closest to the change in the height HL of the fork 16 after time t1 is data D2.

[0059] The control unit 45 refers to the heights HL1 and HL2 of the target opening Pg when the data D1 and D2 are executed. In the estimation process S3 of this embodiment, the control unit 45 estimates the average value of the height HL1 of the target opening Pg when the data D1 is executed and the height HL2 of the target opening Pg when the data D2 is executed as the height Hog of the target opening Pg of this embodiment. Therefore, the estimation process S3 of this embodiment estimates the pallet P including the target opening Pg from the operation history of the lift lever 17.

[0060] <Actions and Effects of the Present Embodiment> According to this embodiment, basically the same actions and effects as those of the first embodiment can be achieved, and the following effects can also be obtained.

[0061] (2-1) Executing the estimation process S3 from the operation history of the lift lever 17 means executing the estimation process S3 from many operation modes of the lift lever 17. This improves the estimation accuracy of the estimation process S3. As a result, the lift assistance process S5 can be executed appropriately.

[0062] [Example of change] The above-described embodiments can be modified as follows: The above-described embodiments and the following modifications can be combined with each other to the extent that no technical contradiction occurs.

[0063] Although the trained model M1 outputs a class probability that is an index of the height Ho of the socket P1 included in the input image data, this is not limited to this. For example, the trained model M1 may be modified to output the height Ho of the socket P1 included in the input image data itself.

[0064] The control unit 45 may calculate the rate of change of the ascending speed of the forks 16 from time t1 to time t1'. Then, when the rate of change of the ascending speed of the forks 16 continues after time t1', the control unit 45 estimates the height Ho of the insertion port P1 acquired in the processing of step S2. The estimated height Ho may be, for example, the insertion port P1 located at the height that is realized earliest among the heights H0, H1, and H2, as the target port Pg. It is sufficient for the control unit 45 to be able to estimate the pallet P having the target port Pg into which the forks 16 are inserted from the amount of operation of the lift lever 17 in a predetermined time Δt.

[0065] The control unit 45 may calculate an average value of the ascending speed of the forks 16 from time t1 to time t1'. Then, when the average value of the ascending speed of the forks 16 continues after time t1', the control unit 45 estimates the height Ho of the insertion port P1 acquired in the processing of step S2. The estimated height Ho may be, for example, the insertion port P1 located at the height that is realized earliest among the heights H0, H1, and H2, as the target port Pg. It is sufficient for the control unit 45 to be able to estimate the pallet P having the target port Pg into which the forks 16 are inserted, from the amount of operation of the lift lever 17 in a predetermined time Δt.

[0066] The control unit 45 may execute the lift assist process S5 when the difference ΔH is smaller than the predetermined value ΔW1. That is, the lift assist process S5 may be executed when the difference ΔH is equal to or smaller than the predetermined value ΔW1.

[0067] Even if modified in this way, the control unit 45 controls the hydraulic mechanism 22 in the lift assist process S5 so that the forks 16 rise, thereby causing the forks 16 to face the target opening Pg.

[0068] 13, the control device 41 may execute additional processes. After executing the lift assist process S5, the control device 41 may execute a tilt process S6. After executing the tilt process S6, the control device 41 may execute a correction process S7.

[0069] As shown in Fig. 14, for example, it is assumed that the platform 100 is tilted backward so as to rise toward the front of the forklift 10. The tilting process S6 is a process for tilting the forks 16 backward so that the angle θ of the target opening Pg is relative to the platform 100. In the tilting process S6, the control device 41 controls the hydraulic mechanism 22 so that the forks 16 are tilted backward by the angle θ from a state in which the forks 16 face the target opening Pg. As a result, the inclination of the forks 16 and the angle θ of the target opening Pg coincide with each other.

[0070] 15, when the tilting process S6 is executed while the mounting table 100 is tilted backward, the tip of the fork 16 rises by the amount of the backward tilt of the fork 16. If the amount of rise of the fork 16 is a change amount Δh, the fork 16 needs to be lowered by the change amount Δh in order to make the fork 16 face the target opening Pg again.

[0071] The correction process S7 is a process for lowering the fork 16 by the amount of change Δh so that the fork 16 faces the target opening Pg. In the correction process S7, the control device 41 controls the hydraulic mechanism 22 to lower the fork 16 by the amount of change Δh. As a result, the fork 16 faces the target opening Pg again.

[0072] In the above modification, it is assumed that the platform 100 is tilted backward, but the tilting process S6 and the correction process S7 may be executed even when the platform 100 is tilted forward so as to descend toward the front of the forklift 10. In this case, the tilting process S6 is a process for tilting the forks 16 forward so that the angle θ of the target opening Pg is relative to the platform 100. In the tilting process S6, the control device 41 controls the hydraulic mechanism 22 so that the forks 16 are tilted forward by the angle θ from the state in which the forks 16 face the target opening Pg.

[0073] Furthermore, when the tilting process S6 is executed in a case where the mounting table 100 is tilted forward, the tip of the fork 16 is lowered by the amount of the forward tilt of the fork 16. If the amount of descent of the fork 16 is a change amount Δh, in order to make the fork 16 face the target opening Pg again, it is necessary to raise the fork 16 by the amount of the change amount Δh.

[0074] The correction process S7 is a process for raising the fork 16 by the amount of the change Δh so that the fork 16 faces the target port Pg. In the correction process S7, the control device 41 controls the hydraulic mechanism 22 so as to raise the fork 16 by the amount of the change Δh.

[0075] That is, the tilting process S6 is a process of tilting the forks 16 so that the angle θ of the target opening Pg is relative to the mounting table 100. The correction process S7 is a process of raising and lowering the forks 16 by the amount of change Δh so that the forks 16 face the target opening Pg. The tilting process S6 and the correction process S7 may be executed by the control unit 45 or may be executed by a functional element newly added to the control device 41.

[0076] When performing the tilting process S6, the control device 41 may obtain the angle θ of the target opening Pg based on an index output from the trained model. The trained model in this case may be the trained model M1 described above, but the trained model M1 may also learn the angle θ of the insertion opening P1 from teacher data such as those shown in Figures 5, 6, and 7. When there is a difference between the acquired angle θ of the target opening Pg and the inclination angle of the fork 16, the control device 41 may control the hydraulic mechanism 22 so that the inclination angle of the fork 16 coincides with the angle θ of the target opening Pg. The trained model used to obtain the angle θ of the target opening Pg in the tilting process S6 may be prepared separately from the trained model M1. In addition, in this modified example, the output result of the trained model may be the angle θ of the target opening Pg itself, rather than an index of the angle θ of the target opening Pg.

[0077] By combining the first and second embodiments, the control unit 45 may execute the estimation process S3 based on both the operation of the lift lever 17 being operated and the operation history of the lift lever 17. Specifically, if there is a discrepancy between the height Hog of the target aperture Pg estimated based on the operation of the lift lever 17 in an operated state and the height Hog of the target aperture Pg estimated from the operation history of the lift lever 17, the height Hog of the target aperture Pg may be calculated by multiplying the height Hog of the target aperture Pg estimated based on the operation of the lift lever 17 by a correction value that corrects the discrepancy. Depending on the correction value, the height Hog of the target aperture Pg may be calculated by adding or subtracting it from the height Hog of the target aperture Pg estimated based on the operation of the lift lever 17.

[0078] Furthermore, when comparing the height Hog of the target opening Pg estimated based on the operation of the lift lever 17 with the height Hog of the target opening Pg estimated from the operation history of the lift lever 17, if there is a discrepancy between the values, the larger or smaller value may be used as the height Hog of the target opening Pg. That is, the estimation process S3 only needs to be able to estimate the pallet P including the target opening Pg into which the forks 16 are inserted from at least one of the operation of the lift lever 17 in the operated state and the operation history of the lift lever 17. At least one of the operation of the lift lever 17 in the operated state and the operation history of the lift lever 17 means only the operation of the lift lever 17 in the operated state, or the operation history of the lift lever 17, or both the operation of the lift lever 17 in the operated state and the operation history of the lift lever 17.

[0079] The forklift 10 may be equipped with a side shifter, a fork shifter, and operation levers for operating each of the side shifter and the fork shifter. In this case, it is preferable to store in the auxiliary storage device 51 a trained model that outputs the position of the insertion port P1 included in the image data in the vehicle width direction of the forklift 10, or an index indicating the position. The control device 41 may then execute an estimation process to estimate the pallet P including the target opening Pg, which is the target insertion port P1 into which the fork 16 is to be inserted, from at least one of the operation of the operation lever in the operated state and the operation history of the operation lever. The control device 41 may then execute an auxiliary process to control the side shifter or the fork shifter so that the fork 16 faces the target opening when the difference between the position of the target opening Pg in the vehicle width direction of the forklift 10 acquired based on the output result of the trained model and the actual position of the fork 16 due to the operation of the operation lever becomes equal to or less than a predetermined value indicating the vicinity of the target opening while the operation lever is being operated by the rider.

[0080] Depending on the setting of the predetermined value ΔW1, the lift assist process S5 may be modified as follows: The lift assist process S5 may be a process in which, when the forks 16 are raised by operating the lift lever 17 and the difference ΔH reaches the predetermined value ΔW1 after the forks 16 pass the target opening Pg, the hydraulic mechanism 22 is controlled to lower the forks 16 so that the forks 16 face the target opening Pg.

[0081] The devices functioning as the acquisition unit 44 and the control unit 45 and the auxiliary storage device 51 may be provided at positions different from those of the forklift 10. In this case, a communication device is provided in the forklift 10, and image data captured by the camera 31 is transmitted by the communication device. The acquisition unit 44 receives the image data transmitted from the communication device, and the control unit 45 executes the estimation process S3 from the output result of the trained model M1. The control unit 45 transmits a command to the forklift 10 to execute the lifting assistance process S5 based on the height Hog of the target opening Pg estimated by the estimation process S3. In the forklift 10, the positioning process of the fork 16 with respect to the target opening Pg is executed in response to a command from the control unit 45.

[0082] The acquisition unit 44 and the control unit 45 may be separate devices. In addition, the control device 41 that controls the loading and unloading operations of the forklift 10, such as controlling the hydraulic mechanism 22, and a device that functions as the acquisition unit 44 and the control unit 45 may be separate devices.

[0083] The camera 31 does not have to be attached to the mast 12. For example, as long as the camera 31 can capture an image of the pallet P of the platform 100 and its insertion port P1, it may be provided at a position different from the forklift 10.

[0084] A plurality of cameras 31 may be used. The insertion port P1 may be formed by the pallet P and the mounting surface of the mounting table 100. The platform 100 may be, for example, the bed of a truck.

[0085] The trained model M1 may be a model using an algorithm other than CNN. For example, examples of machine learning algorithms include SSD (Single Shot Multibox Detector), R-CNN (Regional Convolutional Neural Network), fast R-CNN, faster R-CNN, and YOLO (You Only Look Once).

[0086] The trained model M1 may be one that uses instance segmentation or semantic segmentation. ○ The trained model M1 may be generated by an algorithm other than DNN. For example, the trained model M1 may be generated using a support vector machine (SVM). In this case, the control device 41 performs feature extraction. The trained model M1 outputs an index of a class to which the height Ho of the socket P1 included in the image data corresponds, based on the feature obtained by the feature extraction. Examples of the feature extraction method include Histograms of Oriented Gradients (HOG) and Scale-Invariant Feature Transform (SIFT).

[0087] The storage device that stores the trained model M1 may be the storage unit 43. If the forklift 10 is equipped with a height sensor, the height HL of the forks 16 may be calculated based on a signal from the height sensor.

[0088] In the present embodiment, the timing when the camera 31 captures the image of the pallet P and its insertion port P1 is when the forklift 10 stops in front of the platform 100 before loading the pallet P onto the forks 16, but this is not limited to this. The timing when the camera 31 captures the image of the pallet P and its insertion port P1 may be changed as appropriate. [Explanation of symbols]

[0089] 10...forklift, 16...fork, 17...lift lever, 20...lift cylinder, 22...hydraulic mechanism, 30...fork position adjustment device, 31...camera, 44...acquisition unit, 45...control unit, 51...auxiliary memory device, P...pallet, P1...insertion port, Pg...target port, M1...trained model, Ho...insertion port height, Hog...target port height, HL...fork height, ΔH...difference, S3...estimation process, S5...lifting assistance process, D1, D2, D3, D4...data.

Claims

1. A fork position adjustment device that is applied to a forklift including a fork, a lifting device that raises and lowers the fork, and a lift lever that is operated by a rider to operate the lifting device, an acquisition unit that acquires image data including an insertion port and the pallet from a camera, the image data including the insertion port and the pallet, the image data being obtained by assuming that a hole into which the fork is inserted in order to load the pallet onto the fork is an insertion port; A storage device that stores a trained model that outputs the height of the socket included in the image data or an index indicating the height of the socket when the image data is input; A control unit that acquires the height of the insertion slot included in the image data based on the output result of the trained model and controls the lifting device, The control unit is an estimation process for estimating the pallet having a target opening, which is the insertion opening into which the forks are to be inserted, based on at least one of the operation of the lift lever in an operated state and the operation history of the lift lever; a lifting assist process that controls the lifting device so that the forks face the target opening when the difference between the acquired target opening height and the actual height of the forks due to operation of the lift lever becomes equal to or less than a predetermined value indicating the proximity of the target opening while the lift lever is being operated by a rider; and

2. The control unit executes the estimation process based on the operation of the lift lever in the operated state.

2. The fork position adjustment device of claim 1.

3. The control unit executes the estimation process based on an operation history of the lift lever.

2. The fork position adjustment device of claim 1.

Images