Display system

Abstract

To make it easy to confirm a situation ahead of a load.SOLUTION: A display system includes a camera, a control device, and a display part. The camera is arranged so as to image the front of a fork lift from a space between a load placed on a fork and a road surface. When the load is placed on the fork, and a lifting height of the fork is less than a low lifting height threshold, the control device enlarges an extraction range from which an upper range from an upper end to a lower side in the image taken by the camera is removed, and displays the enlarged range on a display.SELECTED DRAWING: Figure 4

Description

Technical Field

[0003] ,

[0001] The present disclosure relates to a display system.

Background Art

[0002] The forklift disclosed in Patent Document 1 includes a camera, a display unit, a control device, and forks. The image captured by the camera is displayed on the display unit. The operator of the forklift can check the situation in front of the forklift by checking the image displayed on the display unit. The control device changes the shooting conditions according to the lifting height of the forks. Thereby, an image corresponding to the lifting height of the forks is displayed on the display unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, when there is a load on the forks, the load enters the image captured by the camera. Since the load stacked on the forks is displayed on the display unit, it is difficult to check the situation in front of the forklift.

Means for Solving the Problems

[0005] A display system for solving the above problems is a display system for displaying the front of a forklift, which is equipped with forks on which a load is placed, a lifting height sensor for detecting the lifting height of the forks, and a load sensor for determining whether or not a load is placed on the forks, and comprises a camera positioned to capture images of the front of the forklift from the space between the load placed on the forks and the road surface, a display unit for displaying the image captured by the camera, and a control device, wherein if a load is placed on the forks and the lifting height is less than a low lifting height threshold, the control device expands the extracted range by excluding the upper range from the top edge downwards from the image and displays it on the display unit.

[0006] When a load is loaded onto the forklift and the forklift's lifting height is below the low-lifting-height threshold, the load is visible in the upper range of the image captured by the camera. If the image including the upper range is displayed on the display unit in this case, the display unit will show the load, making it difficult to see what is in front of the load. When a load is loaded onto the forklift and the forklift's lifting height is below the low-lifting-height threshold, the control device expands the extraction range and displays it on the display unit, reducing the proportion of the display unit occupied by the load and making it easier to see what is in front of the load.

[0007] Regarding the above display system, the control device may raise the position of the upper end of the extraction range as the lifting height increases. Regarding the above display system, the control device may enlarge the extracted range, excluding the downward range extending upward from the lower edge of the image, and display it on the display unit. [Effects of the Invention]

[0008] According to this invention, it becomes easier to check the situation in front of the luggage. [Brief explanation of the drawing]

[0009] [Figure 1] This is a side view of a forklift. [Figure 2]Figure 1 is a front view of the outer mast of the forklift. [Figure 3] Figure 1 is a schematic diagram of the forklift configuration. [Figure 4] Figure 3 is a flowchart showing the display control performed by the control device. [Figure 5] This figure shows the extraction range extracted from the image using the display control in Figure 4. [Figure 6] This figure shows the relationship between the upper end of the extraction range in Figure 5 and the lifting height of the fork. [Figure 7] This is a side view showing a forklift whose fork lifting height is above the low lifting height threshold. [Figure 8] Figure 7 shows the image displayed on the display unit of the forklift. [Figure 9] This figure shows the image displayed on the display unit of the forklift shown in Figure 1. [Modes for carrying out the invention]

[0010] One embodiment of the display system will be described. <Forklift> As shown in Figure 1, the forklift 10 is a counterbalanced forklift. The forklift 10 may also be a reach forklift. The forklift 10 is operated manually by an operator riding on the forklift 10. The forklift 10 may be configured to allow switching between automatic and manual operation. In the following description, front, back, left, and right refer to the front, back, left, and right relative to the forklift 10. The forklift 10 is equipped with a display system DS.

[0011] The forklift 10 comprises a body 11, drive wheels 12, steering wheels 13, and a cargo handling device 14. The cargo handling device 14 is located at the front of the vehicle body 11. The cargo handling device 14 includes a mast 15. The mast 15 includes an outer mast 21 and an inner mast 17.

[0012] As shown in Figure 2, two outer masts 21 are provided, spaced apart from each other in the left-right direction. The cargo handling device 14 includes an outer mast bar 22 that connects the two outer masts 21. The outer mast bar 22 connects the lower ends of the two outer masts 21. Two inner masts 17 are provided, spaced apart from each other in the left-right direction. The inner masts 17 are movable up and down relative to the outer masts 21.

[0013] As shown in Figure 1, the cargo handling device 14 includes a lift bracket 18. The lift bracket 18 is mounted so as to be able to move up and down relative to the inner mast 17. The cargo handling device 14 is equipped with forks 19. The forks 19 are attached to a lift bracket 18. The forks 19 move up and down together with the lift bracket 18. The cargo W1 to be handled is placed on the forks 19.

[0014] The cargo handling device 14 includes a lift cylinder 31. The lift cylinder 31 is a hydraulic cylinder. The lift cylinder 31 raises and lowers the forks 19. The cargo handling device 14 also includes a tilt cylinder 32. The tilt cylinder 32 is a hydraulic cylinder. The tilt cylinder 32 tilts the forks 19.

[0015] As shown in FIG. 3, the display system DS includes a control device 41. The control device 41 includes a processor 42 and a storage unit 43. As the processor 42, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor) is used. The storage unit 43 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). A program for operating the forklift 10 is stored in the storage unit 43. It can be said that the storage unit 43 stores program codes or instructions configured to cause the processor 42 to execute processing. The storage unit 43, that is, the computer-readable medium, includes any available medium that can be accessed by a general-purpose or dedicated computer. The control device 41 may be constituted by a hardware circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control device 41, which is a processing circuit, may include one or more processors operating according to a computer program, one or more hardware circuits such as an ASIC or an FPGA, or a combination thereof.

[0016] The forklift 10 includes a lift sensor 51. The lift sensor 51 detects the lift of the fork 19. The lift of the fork 19 is the height from the road surface to the fork 19. The lift sensor 51 outputs an electrical signal corresponding to the lift of the fork 19 to the control device 41. The control device 41 can recognize the lift of the fork 19 based on the electrical signal from the lift sensor 51.

[0017] The forklift 10 includes a load sensor 52. The load sensor 52 outputs an electrical signal corresponding to the weight of the load W1 placed on the fork 19 to the control device 41. The control device 41 can recognize the weight of the load W1 based on the electrical signal of the load sensor 52. The load sensor 52 is a loading sensor.

[0018] The display system DS includes a camera 53. The camera 53 is a digital camera. The camera 53 includes an image sensor. The image sensor is, for example, a CCD image sensor (Charge Coupled Device image sensor) or a CMOS image sensor (Complementary Metal Oxide Semiconductor image sensor). The camera 53 is positioned to capture images of the area in front of the forklift 10 from the space between the load W1 and the road surface when the load W1 is loaded on the forks 19. For example, as shown in Figure 2, the camera 53 is positioned on the outer mast bar 22.

[0019] The display system DS includes a display unit 61. The display unit 61 is positioned so that it can be seen by the operator of the forklift 10. The display unit 61 is, for example, a liquid crystal display or an organic electroluminescent display.

[0020] The control device 41 updates the display content of the display unit 61. The control device 41 displays the image obtained by the camera 53 on the display unit 61. The display control performed by the control device 41 will now be described. The display control is performed repeatedly at a predetermined control cycle.

[0021] <Display Control> As shown in Figure 4, in step S1, the control device 41 obtains the lifting height of the fork 19 from the lifting height sensor 51.

[0022] Next, in step S2, the control device 41 acquires the load from the load sensor 52. Next, in step S3, the control device 41 determines whether the forklift 10 is holding the load W1 and whether the lifting height of the forks 19 is low. If the determination result in step S3 is negative, the control device 41 performs the process in step S4. If the determination result in step S3 is positive, the control device 41 performs the process in step S5.

[0023] The control device 41 determines that the forklift 10 is holding the load W1 if the weight of the load W1 placed on the forks 19 is equal to or greater than the load determination threshold. The weight of the load W1 placed on the forks 19 can be determined from the detection result of the load sensor 52. The load sensor 52 is provided to determine whether or not a load W1 is placed on the forks 19. The load determination threshold is a predetermined value. The load determination threshold is set to a value that allows the control device 41 to determine that a load W1 is placed on the forks 19. For example, the load determination threshold is set to a value slightly higher than the weight detected by the load sensor 52 when no load W1 is placed on the forks 19. As a result, when a load W1 is placed on the forks 19, the control device 41 determines that the forklift 10 is holding the load W1.

[0024] The control device 41 determines that the lifting height of the fork 19 is low if it is below the low lifting height threshold. The lifting height of the fork 19 can be determined from the detection result of the lifting height sensor 51. The low lifting height threshold is a predetermined value. The lower the lifting height of the fork 19, the higher the proportion of the load W1 that occupies the imaging range of the camera 53. The low lifting height threshold is set so that the proportion of the load W1 that occupies the imaging range of the camera 53 is acceptable. The low lifting height threshold may also be set to a lifting height at which the load W1 does not enter the imaging range of the camera 53 when the load W1 is loaded on the fork 19.

[0025] In step S4, the control device 41 performs normal display. Normal display means displaying the image on the display unit 61 without partially enlarging the image obtained by the camera 53. That is, the control device 41 displays the entire image obtained by the camera 53 on the display unit 61. After completing the process in step S4, the control device 41 terminates the display control.

[0026] In step S5, the control device 41 determines the extraction range. As shown in Figure 5, the extraction range A1 is a part of the image IM1 obtained by imaging with the camera 53. In this embodiment, the extraction range A1 is a range that has the same width dimension as the image IM1 and a shorter height dimension than the image IM1. The width direction of the image IM1 is the X-axis direction in the coordinate system of the image IM1. The height direction of the image IM1 is the Y-axis direction in the coordinate system of the image IM1.

[0027] The upper range A2 is defined as the area of ​​image IM1 extending downwards from the upper edge E1 of image IM1. The upper range A2 is the area from the upper edge E1 of image IM1 to the extraction range A1. The lower edge of the upper range A2 is at the same position as the upper edge E11 of the extraction range A1.

[0028] The lower range A3 is defined as the area extending upward from the lower edge E2 of image IM1. The lower range A3 is the range from the lower edge E2 of image IM1 to the extraction range A1. The upper edge of the lower range A3 is at the same position as the lower edge E12 of the extraction range A1.

[0029] As shown in Figure 6, when the lifting height of the forks 19 is below the low lifting height threshold, the control device 41 raises the position of the upper end E11 of the extraction range A1 as the lifting height increases. As a result, when the lifting height of the forks 19 is below the low lifting height threshold, the higher the lifting height of the forks 19, the shorter the height dimension of the upper range A2 becomes. When the lifting height of the forks 19 is below the low lifting height threshold, the load W1 is captured in the image IM1 because it enters the imaging range of the camera 53. Since the camera 53 is positioned to capture the area in front of the forklift 10 from between the load W1 and the road surface, the load W1 will be captured in the range extending downward from the upper end E1 of the image IM1. The upper range A2 is the range of the image IM1 in which the load W1 is assumed to be captured. Furthermore, as the lifting height of the forks 19 increases, the proportion of the imaging range occupied by the load W1 decreases. Therefore, by raising the position of the upper end E11 of the extraction range A1 as the lifting height of the fork 19 increases, the position of the upper end E11 of the extraction range A1 also increases as the position of the lower end E22 of the load W1 captured in image IM1 increases. In this embodiment, the upper end E11 of the extraction range A1 is set to be located a predetermined distance d1 above the lower end E22 of the load W1 captured in image IM1. The predetermined distance d1 is a margin.

[0030] The lower end E12 of the extraction range A1 is at a constant position. That is, the lower range A3 is a constant range. The position of the lower end E12 of the extraction range A1 is determined, for example, by the length of the fork 19. When a load W1 is placed on the fork 19, the road surface is located below the load W1. The lower range A3 is the area of ​​image IM1 that is assumed to include the road surface.

[0031] As shown in Figure 4, in step S6, the control device 41 then performs an enlarged display. The enlarged display involves enlarging the portion of the image IM1 that excludes the upper range A2 and the lower range A3, i.e., the extracted range A1, and displaying it on the display unit 61. In the enlarged display, the extracted range A1 is stretched in the height direction compared to when the entire image IM1 is displayed on the display unit 61. After completing the process in step S6, the control device 41 terminates the display control.

[0032] [Operation of this embodiment] As shown in Figure 7, the forklift 10 is holding the load W1, and the lifting height of the forks 19 is above the low lifting height threshold. In this case, the control device 41 displays the normal information. Because the lifting height of the forks 19 is above the low lifting height threshold, the load W1 does not enter the imaging range of the camera 53.

[0033] As shown in Figure 8, if an obstacle M1 is present in front of the forklift 10, the display unit 61 will normally display the obstacle M1. Obstacle M1 includes people and non-personal objects. Since the display unit 61 does not display the load W1, the operator of the forklift 10 can check the situation in front of the load W1 by looking at the display unit 61.

[0034] As shown in Figure 1, the forklift 10 is holding a load W1, and the lifting height of the forks 19 is set to a low lifting height. In this case, the load W1 is captured in the upper range A2 of the image IM1 taken by the camera 53. If normal display is performed in this case, the display unit 61 will show the load W1, making it difficult to see the situation in front of the load W1. For example, even if there is an obstacle M1 in front of the load W1, it will be difficult to see the obstacle M1. In this embodiment, when the forklift 10 is holding a load W1 and the lifting height of the forks 19 is set to a low lifting height, the control device 41 performs an enlarged display.

[0035] As shown in Figure 9, if an obstacle M1 is present in front of the forklift 10, the display unit 61 will enlarge and display the obstacle M1 that can be captured from between the load W1 and the road surface. In the example shown in Figure 9, a person's foot captured from between the load W1 and the road surface will be displayed enlarged on the display unit 61.

[0036] [Effects of this embodiment] (1) The control device 41 performs an enlarged display when a load W1 is loaded on the forks 19 and the lifting height of the forks 19 is low. When the control device 41 performs an enlarged display, the proportion of the image captured in front of the load W1 on the display unit 61 is higher than when normal display is performed. The proportion of the display unit 61 occupied by the load W1 is reduced, making it easier for the operator of the forklift 10 to check the situation in front of the load W1. The aspect ratio when the enlarged display is performed will be different from the aspect ratio when the entire image IM1 is displayed on the display unit 61. As a result, the shape of the obstacle M1 displayed on the display unit 61 will be different from the shape of the obstacle M1 when the entire image IM1 is displayed on the display unit 61. However, in terms of the operation of the forklift 10, it is sufficient to determine whether or not an obstacle M1 exists in front of the load W1. For this reason, changes in the shape of the obstacle M1 are acceptable.

[0037] (2) The control device 41 raises the upper end E11 of the extraction range A1 as the lifting height of the forks 19 increases. As the lifting height of the forks 19 increases, the proportion of the load W1 in the imaging range decreases. As the lifting height of the forks 19 increases, the height dimension of the upper range A2 is shortened, which widens the range between the load W1 and the road surface that is displayed on the display unit 61. This makes it easier for the operator of the forklift 10 to check the situation in front of the load W1.

[0038] (3) In the enlarged view, the extracted range A1, which excludes the lower range A3 in addition to the upper range A2, is enlarged and displayed on the display unit 61. The lower range A3 is the range in the image IM1 that is assumed to include the road surface located below the cargo W1. By excluding the lower range A3, the proportion of the display unit 61 that captures the area in front of the cargo W1 increases. This makes it easier for the operator of the forklift 10 to check the situation in front of the cargo W1.

[0039] [Example of changes] The embodiment can be implemented with the following modifications. The embodiment and the following modifications can be combined with each other to the extent that they do not contradict each other technically.

[0040] ○When the control device 41 performs enlarged display, it may display on the display unit 61 an extracted range A1 which excludes only the upper range A2 from the image IM1. In other words, the control device 41 does not need to exclude the lower range A3 from the image IM1.

[0041] ○The control device 41 does not need to change the position of the upper end E11 of the extraction range A1 based on the lifting height of the fork 19. In other words, the extraction range A1 may be a fixed range. ○The display unit 61 may be equipped with a touch panel. In this case, the position of the upper end E11 of the extraction range A1 may be changed by pinch-in or pinch-out operations.

[0042] ○The control device 41 may maintain multiple correspondences between the lifting height of the fork 19 and the position of the upper end E11 of the extraction range A1. Furthermore, when the position of the upper end E11 of the extraction range A1 is raised as the lifting height of the fork 19 increases, the control device 41 may determine the position of the upper end E11 of the extraction range A1 according to the selected correspondence.

[0043] ○As a load sensor, a switch that turns on and off depending on whether or not a load W1 is placed on the fork 19 may be used. Alternatively, a distance meter that measures the distance above the fork 19 may be used as a load sensor. When a load W1 is placed on the fork 19, the distance measured by the distance meter becomes shorter, so the control device 41 can determine whether or not a load W1 is placed on the fork 19.

[0044] ○The forklift 10 may be equipped with a device that can detect the height of the lower end of the load W1. Such a device may be, for example, a camera, LIDAR (Laser Imaging Detection and Ranging), or millimeter-wave radar. The control device 41 may set the height dimension of the upper range A2 based on the height of the lower end of the load W1 so that the load W1 is not displayed on the display unit 61.

[0045] ○The forklift 10 may be remotely operated. In this case, the operator operates the forklift 10 from a remote location away from the forklift 10. The operator operates a remote control device installed at the remote location. A dedicated device may be used as the remote control device, or a portable communication terminal such as a tablet terminal may be used. The remote control device comprises an operating unit for operating the forklift 10, a communication device, a terminal control device, and a terminal display unit. The communication device is configured to communicate with the forklift 10. The terminal control device can acquire the image IM1 captured by the camera 53 via the communication device. The terminal control device displays the image IM1 on the terminal display unit by performing display control similar to that of the embodiment. In this case, the terminal control device is the control device. The terminal display unit is the display unit. The forklift 10 and the remote control device constitute the display system. [Explanation of Symbols]

[0046] A1... Extraction range, A2... Upper range, A3... Lower range, DS... Display system, E1... Top edge, E2... Bottom edge, E11... Top edge, IM1... Image, W1... Cargo, 10... Forklift, 19... Fork, 41... Control device, 51... Lifting height sensor, 52... Load sensor (which is a load sensor), 53... Camera, 61... Display unit.

Claims

1. A fork that can carry luggage, A lifting height sensor for detecting the lifting height of the fork, A display system for displaying the front of a forklift, comprising a load sensor for determining whether or not the load is loaded on the forks, A camera positioned to capture images of the area in front of the forklift from the space between the load placed on the forks and the road surface, A display unit that displays the image captured by the aforementioned camera, A control device is provided, When the load is placed on the fork and the lifting height is below the low lifting height threshold, the control device expands the extracted range by excluding the upper range from the top edge downwards from the image and displays it on the display unit. A display system that raises the position of the upper end of the extraction range as the height of the aforementioned lift increases.

2. A fork that can carry luggage, A lifting height sensor for detecting the lifting height of the fork, A display system for displaying the front of a forklift, comprising a load sensor for determining whether or not the load is loaded on the forks, A camera positioned to capture images of the area in front of the forklift from the space between the load placed on the forks and the road surface, A display unit that displays the image captured by the aforementioned camera, A control device is provided, The control device is a display system that, when the load is placed on the fork and the lifting height is less than a low lifting height threshold, expands the extracted range by excluding the upper range from the top edge downwards and the lower range from the bottom edge upwards of the image from the image and displays it on the display unit.

Images