Remote forklift operating system

Abstract

[Problem] To improve the safety assurance and the accuracy of remotely operated forklift work by providing, to an operator who remotely operates a forklift, an image for performing the confirmation work and assessment that are prescribed and necessary when operating the forklift. [Solution] A remote forklift operating system 10 comprising a forklift 101 and a remote operation device 201 that can communicate with each other, wherein: the forklift 101 includes one or more cameras 105, 106 for which the positions and wide angles thereof are set so as to capture, during unloading, the status of a cargo placement site Y, the orientation relationship between the front of the forklift body and the cargo placement site Y, an area including the clearance between the cargo placement site Y and a left and right pair of outriggers 111, the state of a loading / unloading target C, the positional relationship between the forks 113 of the forklift 101 and fork insertion holes of the loading / unloading target C, and the rearward periphery of the forklift body; and the remote operation device 201 includes a display unit 203 for displaying the images captured by the one or more cameras 105, 106 on a screen.

Description

Forklift remote control system

[0001] The present invention relates to a forklift remote control system for remotely operating a forklift.

[0002] Advances in wireless communication systems have made it possible to provide high-quality wireless networks with high capacity and low latency. As a result, remote applications that require high communication quality, such as remote medical services and remote robot control, which were previously difficult to achieve, are now being considered. The logistics industry has also responded to this trend, and is considering remote forklifts that can be used to alleviate labor shortages by providing support at multiple locations, as well as for use in harsh environments such as refrigerated and frozen warehouses.

[0003] To ensure safe and efficient operation of a remote-controlled forklift, it is important for the operator to accurately grasp the on-site environment from a remote location. A conventional technology for this is a driving assistance device for an industrial vehicle, as disclosed in Patent Document 1. The driving assistance device for an industrial vehicle disclosed in Patent Document 1 is equipped with four cameras mounted on the left and right front ends and the left and right rear ends of the base of the industrial vehicle to capture images of the four corners of the vehicle, and creates and displays an overhead image of the area around the vehicle from the camera images captured by the four cameras. This improves the visibility of interfering objects around the vehicle.

[0004] Japanese Patent Application Laid-Open No. 2020-040750

[0005] However, the driving assistance device for industrial vehicles disclosed in Patent Document 1 only provides an overhead image of the area around the vehicle, and does not provide images that allow the operator to perform confirmation tasks required for remotely operating a forklift, such as images to confirm whether the forks can be inserted into the pallet pockets when loading cargo.

[0006] Therefore, the present invention aims to provide a remote operation system, a forklift remote operation method, and a program that can provide an operator remotely operating a forklift with images to perform the confirmation work and judgment that are specified and necessary when operating the forklift, thereby ensuring the safety and improving the accuracy of remotely operated forklift operations.

[0007] A representative example of the invention disclosed in this application is as follows. a forklift remote operation system including a forklift capable of communicating with each other and having a holding member for holding an object to be handled, and a remote operation device, wherein the forklift is equipped with one or more cameras whose installation position and imaging range are set so as to photograph, when the object to be handled, loaded on the holding member onto one shelf of a rack having a plurality of shelves arranged one above the other, before unloading: the state of the one shelf; the facing relationship between the front of the forklift body and the one shelf and / or the facing relationship between the front of the forklift body and the rack; a portion including a clearance between the one shelf and the front end of the forklift body and / or a portion including a clearance between the rack and the front end of the forklift body; an appearance of the loaded object to be handled, including clearances on both the top and bottom and the left and right sides between the one shelf; the positional relationship between the holding member and the object to be handled; and, after unloading: the area around the rear of the forklift body and the area below the holding member as the holding member is lowered; and the remote operation device The forklift remote control system is provided with a display unit that displays images captured by the one or more cameras on a screen.

[0008] According to the present invention, images are provided to the operator remotely operating a forklift to enable them to perform the confirmation work and judgment that are specified and necessary when operating the forklift, thereby ensuring the safety and improving the accuracy of remotely operated forklift operations.

[0009] FIG. 1 is a diagram illustrating an example configuration of a forklift according to the present embodiment. FIG. 1 shows an example of the camera positions and camera fields of view of cameras located on the front left and right sides according to the present embodiment. FIG. 2 shows an example of a camera image of cameras located on the front left and right sides according to the present embodiment. FIG. 3 shows another example of the camera positions and camera fields of view of cameras located on the front left and right sides according to the present embodiment. FIG. 4 shows another example of a camera image of cameras located on the front left and right sides according to the present embodiment. FIG. 5 shows an example of the camera position and camera field of view of a camera located at the front lower center according to the present embodiment. FIG. 6 shows an example of a camera image of a camera located at the front lower center according to the present embodiment. FIG. 7 shows another example of the camera position and camera field of view of a camera located at the front lower center according to the present embodiment. FIG. 8 shows an example of a camera image of a camera located at the front center according to the present embodiment. FIG. 9 shows another example of the camera position and camera field of view of a camera located at the front center according to the present embodiment. FIG. 10 shows an example of a camera image of a camera located at the front center according to the present embodiment. FIG. 11 shows an example of a camera image of a camera located at the front center according to the present embodiment. 1 shows example camera images from cameras located on the left and right of the rear according to this embodiment; FIG. 2 shows an example of the camera position and camera field of view of a camera located at the center of the rear according to this embodiment; FIG. 3 shows an example of a camera image from a camera located at the center of the rear according to this embodiment; FIG. 4 is a block diagram showing the functional configuration of a forklift remote control system according to this embodiment; FIG. 5 is a diagram showing an example of the hardware configuration of a forklift and a controller according to this embodiment; FIG. 6 is a flowchart showing image processing by an image processing unit according to this embodiment; FIG. 7 shows example screen displays and example image arrangements of a rear display screen B1 and a front display screen F1 when the forklift is traveling in reverse according to this embodiment; FIG. 8 shows example screen displays and example image arrangements of a rear display screen B2 and a front display screen F3 when the forklift is traveling forward according to this embodiment; FIG. 9 is an explanatory diagram of processing when combining a forward display screen F3; FIG. 10 shows example screen displays and example image arrangements of a rear display screen B2 when the forklift is operating at a low level and a front display screen F1 when stopped according to this embodiment.10A and 10B are diagrams illustrating examples of screen display and image arrangement of a rear display screen B2 when a forklift according to the present embodiment is performing high-place loading and unloading and a front display screen F2 when the forklift is stopped. 10B and 10C are diagrams illustrating processing when the front display screens F1 and F2 when the forklift is stopped are combined.

[0010] Embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings represent the same components.

[0011] Furthermore, when it is not necessary to separately describe a component, it will be described without a subscript (for example, communication unit 108), and when it is necessary to separately describe a component, it will be described with a subscript (for example, communication unit 108-A). Furthermore, in this specification and drawings, left and right and front and rear are defined with the tip of the fork of the forklift being the front.

[0012] The forklift in this invention is a general term for a self-propelled material handling and transport vehicle equipped with a mast that raises and lowers forks, as defined in JIS D 6201, such as a reach forklift, a counter-load forklift, and a three-way forklift. The forklift in this invention is composed of a body and a holding unit that holds the object to be handled. The holding unit is not limited to two forks, and may be various attachments depending on the purpose of use and the shape of the object to be handled. Examples of attachments include a side shifter, a fork shifter, a hinged fork, a rotating fork, a fork multi-load hand roller, a bale clamp or roll clamp, a drum clipper, a push-pull, a bucket, and a ram.

[0013] <Example of Forklift Configuration> In this embodiment, the forklift 101 in the forklift operation assistance system 10 is a reach forklift (hereinafter simply referred to as the forklift 101) that is comprised of a body and two fork holders and can be operated both remotely and from inside. An example of the configuration of the forklift 101 will be described with reference to FIG. 1 . Note that the forklift 101 is not limited to the example configuration shown in FIG. 1 . FIG. 1 is a perspective view showing an example of the configuration of a forklift in the remote operation system according to this embodiment. FIG. 1( a) is a left-front perspective view of the forklift 101, and FIG. 1( b) is a left-rear perspective view of the forklift 101. In this embodiment, the forklift 101 is a reach forklift, but it may also be a counter-load forklift or a 3-way forklift. Of course, other types of forklifts may also be used.

[0014] The forklift 101 comprises a base 110 equipped with a driver's seat 140, a body consisting of outriggers 111, front wheels 112, rear wheels 114, caster wheels (training wheels) 115, a mast 120, a backrest 121, and a head guard 130, and a holder for the forks 113. A pair of left and right outriggers 111a and 111b (referred to simply as "outriggers 111" when referring to both of the pair of left and right outriggers) are disposed on the front side of the base 110, and each of the pair of outriggers 111 extends forward. In this embodiment, the tip of the foremost outrigger 111 on the body of the forklift 101 is the tip of the body. More specifically, the outrigger 111a is provided on the right side, and the outrigger 111b is provided on the left side. Front wheels 112a and 112b are disposed in front of the outriggers 111a and 111b, respectively. Specifically, the right front wheel 112a is provided on the right outrigger 111a, and the left front wheel 112b is provided on the left outrigger 111b. In this manner, a pair of left and right front wheels 112a, 112b are provided on the front side of the machine base 110.

[0015] Rear wheels 114 and caster wheels (auxiliary wheels) 115 are arranged at the rear of the machine base 110. The rear wheels 114 are provided on the left side of the machine base 110, and the caster wheels 115 are provided on the right side of the machine base 110. The rear wheels 114 are drive wheels and steering wheels.

[0016] The forklift 101 runs on three wheels: two front wheels 112a, 112b and one rear wheel 114. A travel motor (not shown) that serves as the drive source for the forklift 101 and a battery (not shown) that serves as the power source for the travel motor are mounted on the machine base 110. The rear wheels 114 are rotated and driven by the travel motor to move the forklift 101 forward and backward.

[0017] The forklift 101 has forks 113a and 113b (hereinafter, referred to as forks 113 when referring to both forks), a backrest 121, and a mast 120 disposed in front of a base 110. Specifically, the mast 120 is attached to the front of the base 110, and the forks 113 and backrest 121 are attached to the mast 120, and they rise and fall integrally with the mast 120. In this embodiment, the front of the vehicle body is the backrest 121. The mast 120 can also tilt forward and backward, and move forward and backward (reach out / in) along the direction in which the outriggers 111 extend.

[0018] The head guard 130 is a roof that covers the cockpit, and is composed of a pair of two support pillars, one on the left and one on the right, attached to the front of the base 110, and a ceiling portion supported by the two support pillars.

[0019] The forklift 101 of this embodiment can be operated not only remotely but also from inside the vehicle. For this purpose, the operator's seat 140 is equipped with a steering wheel for steering the wheels 114, a lift lever and tilt lever for operating the mast 120, operation levers such as a forward / reverse switch lever, a direction lever for driving the forklift 101, and operation pedals operated with the feet, such as a clutch pedal, a brake pedal, and an accelerator pedal (none of which are shown).

[0020] The forklift 101 is equipped with cameras (hereinafter referred to as front cameras) 105-1, 105-2, 105-3, 105-4, and 105-5 at its front, and cameras (hereinafter referred to as rear cameras) 106-1, 106-2, and 106-3 at its rear. The configuration shown in this embodiment, which includes front cameras 105-1 to 105-5 (hereinafter, 105 will be used to collectively refer to the front cameras) and rear cameras 106-1 to 106-3 (hereinafter, 106 will be used to collectively refer to the rear cameras), is just one example, and as long as it can capture images that allow the operator to perform confirmation work while operating the forklift 101, i.e., areas to be confirmed and / or judged (confirmation areas), the forklift 101 may be equipped with a single front camera and a single rear camera, or may have any number of front cameras and rear cameras, each of which may be one or more. Furthermore, the mounting positions, camera types, and imaging ranges of the front camera 105 and rear camera 106 shown in this embodiment are also examples, and the mounting positions, camera types, and imaging ranges of the cameras can be set arbitrarily as long as the area to be checked can be photographed.

[0021] The front camera 105-1 is a wide-angle camera attached to the front right of the forklift 101. More specifically, the front camera 105-1 is attached to the lower right side of the backrest 121 so as to face downward. It is desirable that the camera 105-1 protrudes outward from the right side of the backrest 121 and further protrudes outward from the outrigger 111a. Because the camera 105-1 is attached to the backrest 121, it moves up and down together with the backrest 121.

[0022] The camera 105-2 is a wide-angle camera that is attached to the front left of the forklift 101, symmetrically with the front camera 105-1 across the backrest 121. More specifically, the front camera 105-2 is attached to the lower left side of the backrest 121 so as to face downward. It is desirable that the camera 105-2 also protrudes outward from the left side of the backrest 121 and further protrudes outward from the outrigger 111b. Like the camera 105-1, the camera 105-2 is attached to the backrest 121 and therefore moves up and down together with the backrest 121.

[0023] 2 and 4 show examples of the camera positions and camera fields of view of the above-mentioned front cameras 105-1 and 105-2, and FIGS. 3 and 5 show examples of camera images of the front cameras 105-1 and 105-2.

[0024] 2A shows a right side view of the forklift 101 and the flat-laid object C, (b) a top view of the forklift 101 and the flat-laid object C, and (c) a left side view of the forklift 101 and the flat-laid object C during loading, with the camera field of view indicated by dashed lines. As shown in the figure, the field of view of cameras 105-1 and 105-2 covers the left and right periphery in front of the outriggers 111, the object C, and the loading area Y. Here, the object C is composed of a pallet P and cargo L placed on the pallet P. The pallet P is provided with fork pockets (also called fork insertion holes, not shown) on both sides of the surface on which the cargo is placed, into which forks can be inserted.

[0025] Fig. 3 shows examples of camera images taken by the front cameras 105-1 and 105-2 within the field of view shown in Fig. 2. In Fig. 3, (a) is the camera image taken by the front camera 105-1, and (b) is the camera image taken by the front camera 105-2.

[0026] Figure 4 shows a state in which the fork 113 has risen to the height of the object C placed on the shelf and the forklift 101 approaches the object C during loading, with (a) being a right side view of the object C placed on the shelf and the forklift 101, Figure 3(b) being a top view of the object C placed on the shelf and the forklift 101, and Figure 3(c) being a left side view of the object C placed on the shelf and the forklift 101, with the camera field of view indicated by a dashed line.

[0027] Here, the shelf S on which the cargo handling object C is placed comprises at least a vertically extending support frame and a horizontally extending beam, with the left, right, front, and rear support frames connected by the beams. A rack R is made up of multiple shelves S connected vertically, or in other words, multiple shelves S arranged one above the other on a rack R. Furthermore, multiple racks R can be connected horizontally, and a specific example of such a rack R is a pallet rack.

[0028] As shown in the figure, the field of view of cameras 105-1 and 105-2 is the front left and right surroundings including the support frames that make up the row of shelves S on which the cargo objects C are placed and the beams connected to the support frames.

[0029] Fig. 5 shows examples of camera images taken by the front cameras 105-1 and 105-2 within the field of view shown in Fig. 4. In Fig. 4, (a) is the camera image taken by the front camera 105-1, and (b) is the camera image taken by the front camera 105-2.

[0030] 1, the front camera 105-3 is a wide-angle camera that is attached to the center of the lower front of the forklift 101, i.e., between the left and right forks 113a, 113b. More specifically, the camera 105-3 is attached to the backrest 121 between the base of the right fork 113a and the base of the left fork 113b so as to face downward. Because the camera 105-3 is attached to the backrest 121, it moves up and down as the forks 113 move up and down.

[0031] 6 and 8 show examples of the camera position and field of view of the camera 105-3, which is located in the center of the lower front section. Note that because camera 105-3 is located almost directly below camera 105-4, camera 105-4 is omitted from Figures 6 and 8. Also, Figures 7 and 9 show example camera images from front cameras 105-3 and 105-4.

[0032] Figure 6 shows the state in which the forklift is stopped in front of the flat-laid cargo object C during loading, with (a) being a right side view of the flat-laid cargo object C and the forklift 101, (b) being a top view of the flat-laid cargo object C and the forklift 101, and (c) being a left side view of the flat-laid cargo object C and the forklift 101, with the camera field of view indicated by a dashed line.

[0033] As shown in Fig. 6, the field of view of the camera 105-3 is the lower front central periphery including the forks 113, the pallet P, and the pallet P with the fork pockets. Fig. 7 is an example of a camera image taken by the front camera 105-3 in the field of view shown in Fig. 6.

[0034] 8A shows a right side view of the forklift 101 and the object C placed on the shelf, with the fork 113 raised to the height of the object C placed on the shelf and the forklift 101 approaching the object C during loading, (a) is a right side view of the object C placed on the shelf, (b) is a top view of the object C placed on the shelf and the forklift 101, and (c) is a left side view of the object C placed on the shelf and the forklift 101, with the camera field of view indicated by a dashed line. As shown in FIG. 8, the field of view of the camera 105-3 is the lower front center area including the outriggers 111 and the underside of the row of shelves S on which the object C is placed. FIG. 9 is an example of a camera image captured by the front camera 105-3 within the field of view shown in FIG. 8.

[0035] Returning to Figure 1, the front camera 105-4 is a fisheye camera attached to the center of the front of the forklift 101. More specifically, the front camera 105-4 is attached to an arm or the like connected to the mast 120 so as to be positioned above the backrest 121 and between the masts 120. The camera 105-4 is preferably attached at the height of the operator's eye level when the operator is in the cockpit and performing operations. Furthermore, because the camera 105-4 is attached to an arm or the like connected to the mast 120, it rises and falls as the mast 120 extends or contracts.

[0036] 10 and 12 show examples of the camera position and field of view of the camera 105-4 located at the front center. FIG. 10 shows a state in which the forklift truck is stopped in front of a flat-laid object C during loading: (a) a right side view of the forklift truck 101 and the flat-laid object C, (b) a top view of the forklift truck 101 and the flat-laid object C, and (c) a left side view of the forklift truck 101, with the camera's field of view indicated by a dashed line. As shown in FIG. 10, the field of view of the camera 105-4 is the front center area including the outriggers 111, the forks 113, and the object C, e.g., near and far from the front of the machine base 110 at 270°C. FIG. 11 shows an example of a camera image captured by the front camera 105-4 within the field of view shown in FIG. 10.

[0037] Figure 12 shows the state in which the fork 113 has risen to the height of the object C placed on the shelf and the forklift 101 approaches the object C during loading, with (a) being a right side view of the object C placed on the shelf and the forklift 101, Figure 3(b) being a top view of the object C placed on the shelf and the forklift 101, and Figure 3(c) being a left side view of the object C placed on the shelf and the forklift 101, with the camera field of view indicated by a dashed line.

[0038] As shown in Fig. 12, the field of view of camera 105-4 is the front center periphery including the upper and lower beams of shelf S, forks 113, and outriggers 111. When an object C to be handled is loaded onto forks 113, the lower field of view of camera 105-4 is blocked by object C, and the front center periphery only includes the upper part of object C to be handled and the upper beam of shelf S. Fig. 13 is an example of a camera image captured by front camera 105-4 within the field of view shown in Fig. 12.

[0039] Returning to FIG. 1 , camera 105-5 is a wide-angle camera attached to the front left of forklift 101. However, camera 105-5 may also be attached to the front right of forklift 101. Specifically, camera 105-5 is attached to outrigger 111b so as to face downward. Like camera 105-2, camera 105-5 preferably protrudes outward from the left side of backrest 121 and further protrudes outward from outrigger 111b. However, camera 105-5 does not have to be directly below camera 105-2, and may be offset in front of or behind camera 105-2. In this way, camera 105-5 is attached to outrigger 111b, and therefore, unlike camera 105-2, does not move up or down even when backrest 121 moves up or down.

[0040] FIG. 14 shows an example of the camera position and field of view of the camera 105-5 located on the left side of the front lower portion. FIG. 14 shows (a) a left side view of the forklift 101 and the flat-laid object C, and (b) a top view of the forklift 101 and the flat-laid object C, with the forklift stopped in front of the object C during loading. The camera's field of view is indicated by a dashed line. As shown in FIG. 14, the field of view of the camera 105-5 is the front area from the diagonally lower left to the right, including the outriggers 111, the forks 113, the pallet P with the pallet pockets, and the loading area Y. Because the camera 105-5 does not move up or down even when the backrest 121 is raised or lowered, even if the loading object C is stored on a shelf, the loading area changes from the flat-laid object to the shelf S, but the field of view does not change. FIG. 15 shows an example of a camera image captured by the front camera 105-5 within the field of view shown in FIG. 14.

[0041] Returning to FIG. 1 , the rear camera 106-1 is a fisheye camera attached to the rear left of the forklift 101. Specifically, the rear camera 106-1 is attached to the rear left corner of the head guard 130, facing downward. The rear camera 106-2 is a fisheye camera attached to the rear right of the forklift 101. Specifically, the rear camera 106-2 is attached to the rear right corner of the head guard 130, facing downward.

[0042] FIG. 16 shows an example of the camera positions and camera fields of view of the cameras 106-1 and 106-2 located on the left and right rear sides. In FIG. 16, (a) is a left side view of the forklift 101, (b) is a top view of the forklift 101, and (c) is a right side view of the forklift 101, with the camera field of view indicated by dashed lines. As shown in FIG. 16, the field of view of the rear cameras 106-1 and 106-2 is the rear surroundings of the platform 110 from above, for example, near and far from 270° C. behind the platform 110. Image data (hereinafter simply referred to as "image") captured by the rear cameras 106-1 and 106-2 having such field of view can be used by the image processing unit 107, described later, to generate an overhead image of the rear surroundings including the platform 110. FIG. 17 shows an example of camera images captured by the rear cameras 106-1 and 106-2 within the field of view shown in FIG. 16.

[0043] Returning to FIG. 1 , the rear camera 106-3 is a fisheye camera attached to the rear center of the forklift 101. More specifically, the rear camera 106-3 is attached to the rear center of the base 110, in other words, the rear center of the operator's seat 140. By attaching the rear camera 106-3 to the rear center, it is possible to capture images from the operator's line of sight when boarding the forklift. The rear camera 106-3 may also be attached to the rear center of the head guard 130, in which case it can be directed downward to capture images of the area behind the forklift 101.

[0044] FIG. 18 shows an example of the camera position and field of view of the rear camera 106-3 described above. In FIG. 18, (a) is a left side view of the forklift 101, and (b) is a top view of the forklift 101, with the camera field of view indicated by a dashed line. The right side view is the same as the left side view and is therefore omitted. As shown in FIG. 18, the field of view of the rear camera 106-3 is the area around the front rear, and includes the distant area. FIG. 19 is an example of a camera image captured by the rear camera 106-3 within the field of view shown in FIG. 18.

[0045] Next, the locations to be checked that are photographed by the front cameras, which are made up of the front cameras 105-1 to 105-5 in this embodiment, and the rear cameras, which are made up of the rear cameras 106-1 to 106-3 in this embodiment, will be described.

[0046] In order to provide the operator with images that enable confirmation and / or judgment during loading and unloading by the forklift 101 and while the forklift is traveling, the installation position and imaging range of the front camera are set so as to capture images of locations (confirmation locations) where confirmation and / or judgment are performed as determined and required during operation. Similarly, the installation position and imaging range of the rear camera are set so as to capture images of locations (confirmation locations) that are determined and required during operation. This allows the operator to properly perform confirmation during loading and unloading by the forklift 101 and while the forklift is traveling, even when the forklift is remotely operated, thereby improving safety and efficiency.

[0047] [Loading] First, we will explain the areas to be checked when loading cargo, which are photographed by the front camera or the rear camera. Loading refers to a state in which a series of operations are being performed to load cargo objects C onto the forklift 101. The operations differ slightly depending on whether the cargo objects C are placed flat or on shelves S of the rack R, as described below.

[0048] <Flat Loading Operation> When the forklift 101 loads a load object C that has been placed flat (hereinafter referred to as flat loading), a series of specific operations are performed in the following order: M1 "Stop": The forklift 101 stops in front of the load object C. M2 "Reach out": The forklift 101 approaches the load object C from its stopped position while inserting the forks 113 into the fork pockets of the load object C. M3 "Reach out": The forklift 101 pushes the forks 113 out from the base 110 and approaches the load object C further. M4 "Pick up": The forklift 101 raises the forks 113 to pick up the load object C. M5 "Reach in": The forklift 101 pulls the forks 113 that have picked up the load object C toward the base 110. M6 "Tilt": The forklift 101 tilts the forks 113 slightly backward, which have been pulled toward the base 110. After M6 "Tilt", the forklift 101 can move forward and backward.

[0049] <Shelf Loading Operation> When the forklift 101 loads an object C stored on a shelf S of the rack R (hereinafter referred to as shelf loading), the series of specific operations are performed in the following order. Operations with the same name have the same content as flat loading. M11 "Stop": The forklift 101 stops in front of the rack R (object C). M12 "Raise": The forklift 101 raises the forks 113 to the height of the shelf S, specifically, to the height of the fork pockets of the object C. M13 "Reach Out": The forklift 101 approaches the rack R (object C) from the stopped position while inserting the forks 113 into the fork pockets of the object C. M14 "Reach Out": The forklift 101 moves the forks 113 out from the base 110 and approaches the object C. M15 "Pick up": The forklift 101 raises the forks 113 to pick up the object C. M16 "Reach in": The forklift 101 pulls the forks 113 that have picked up the object C toward the machine base 110. M17 "Reverse": The forklift 101 retreats to a position where the object C will not hit the rack R. M18 "Lower": The forklift 101 lowers the forks 113 that are at the height of the shelf S. The lowering height is a height where the bottom of the loaded object C does not touch the top surface of the outriggers 111, for example, about 20 cm above the outriggers 111. M19 "Tilt": The forklift 101 tilts the lowered forks 113 slightly backward. After M19 "Tilt", the forklift 101 can move forward or backward.

[0050] As described above, since the operations differ in part between flat loading and shelf loading, the points to be checked also differ in part. Therefore, the points to be checked will be explained separately for flat loading and shelf loading.

[0051] <Points to Check When Loading a Load Flat> When loading a load flat, the first point to be checked by the front camera 105 is the front of the machine, i.e., the backrest 121 in the case of the forklift 101 of this embodiment, and the object C to be loaded. A direct facing relationship means that the object C to be loaded is located directly in front of the front of the machine, i.e., the backrest 121 in the case of the forklift 101 of this embodiment, and the backrest 121 and the object C to be loaded are approximately parallel to each other. During M1 "Stop," the operator checks this direct facing relationship to confirm that the forklift 101 has stopped in a position where the object C to be loaded can be safely loaded. The presence of the object to be loaded directly in front of the backrest 121 can also be confirmed by checking that the object is located between the outriggers 111a and 111b.

[0052] When loading flat loads, the second check point photographed by the front camera 105 is the tip of the machine body, that is, in the case of the forklift 101 of this embodiment, the part including the clearance between the tip of the outrigger 111 and the load object. In this embodiment, the clearance may mean either a gap or a distance. When performing M2 "loading," the operator checks that the clearance is such that the clearance between the outrigger 111 and the load object C is minimized. Note that the clearance may be any width, including zero.

[0053] During flat loading, the third check point photographed by the front camera 105 is the positional relationship between the holding unit and the object C to be loaded, i.e., in this embodiment, the positional relationship between the fork 113 and the fork pocket of the object C to be loaded, and confirms that the object C to be loaded is in a position where it can be held by the holding unit. When adjusting the height and left / right of the fork 113 before inserting the fork during M2 "reach out," the operator confirms that the fork 113 can be inserted into the fork pocket of the object C based on the positional relationship. Furthermore, during M2 "reach out" and M3 "reach out," the operator confirms during and after inserting the fork that the fork 113 does not interfere with the object C to be loaded and that it is inserted all the way.

[0054] During flat loading, the fourth check point photographed by the front camera 105 is the state of the object C. Here, the state of the object C includes the state of the object C itself and the state of the surrounding objects. During M4 "pick up," the operator checks the state of the surrounding objects, specifically the height of the object C from the floor. At this time, the operator also checks that the object C is off the ground and has risen to a position higher than the outriggers 111. Furthermore, during M2 "pull up," M3 "reach out," M4 "pick up," M5 "reach in," and M6 "tilt," the operator checks the state of the object C itself, specifically the stability of the object C.

[0055] In this way, the installation position and imaging range of the front camera 105 are determined so that, when loading flat, it photographs the first to fourth check points described above, namely, the direct facing relationship between the front of the machine body (backrest 121) and the object to be loaded C, the area including the clearance between the object to be loaded C and the tip of the machine body (outrigger 111), the positional relationship between the holding part (fork 113) and the holding location (fork pocket) of the object to be loaded C, and the appearance of the object to be loaded C.

[0056] <Check points when loading cargo onto a shelf> When loading cargo onto a shelf, the first check point photographed by the front camera 105 is the first check point when loading cargo flat, the front of the machine body, i.e., the direct facing relationship between the backrest 121 and the cargo object C and / or the direct facing relationship between the backrest 121 and the rack R. Note that since the cargo object C is placed on the shelf S of the rack R, the direct facing relationship between the backrest 121 and the rack R may be the first check point. Furthermore, if a line tape is drawn on the floor parallel to the front of the rack R, the direct facing relationship between the backrest 121 and the line tape may be the first check point.

[0057] The "direct facing" relationship between the backrest 121 and the rack R refers to a relationship in which the rack R is directly in front of the backrest 121 and is approximately parallel to the backrest 121. At the time of M11 "Stop," the operator determines from this direct facing relationship that the forklift 101 has stopped in a position where the object C can be safely loaded. The fact that the rack R is directly in front of the backrest 121 can also be confirmed by checking that the outriggers 111a and 111b are within the width of the rack R. At the time of M11 "Stop," the operator checks the area including the clearance between the fork 113 and the rack R, which is photographed by the front camera 105, in addition to the first check point and the second check point described below. This is because the tip of the fork 113 needs to be in front of the front end of the rack R in order to raise the fork 113 at M12.

[0058] The second check point photographed by the front camera 105 during shelf loading is the second check point during flat loading, the tip of the vehicle, i.e., in the case of the forklift 101 of this embodiment, the area including the clearance between the tip of the outrigger 111 and the object C to be loaded, and / or the area including the clearance between the outrigger 111 at the tip of the vehicle and the rack R. Note that since the object C to be loaded is placed on the shelf S of the rack R, only the area including the clearance between the outrigger 111 and the rack R may be set as the second check point. Furthermore, if a line tape is drawn on the floor with a predetermined clearance from the rack R, the area including the clearance between the outrigger 111 and the line tape may be set as the second check point.

[0059] When M13 "lifting", the operator checks that the clearance between the outrigger 111 and the rack R with the shelf S ensures that the outrigger 111 and the rack R, more specifically, the outrigger 111 and the support frame and beam that make up the rack R, are not in contact.

[0060] During shelf loading, the third check point photographed by the front camera 105, similar to the third check point during flat loading, is the positional relationship between the holding unit and the object C to be handled, i.e., in this embodiment, the positional relationship between the fork 113 and the fork pocket of the object C, and confirms that the object C is in a position where it can be held by the holding unit. During shelf loading, the fork 113 is raised by M12 "up." Therefore, before inserting the fork during M12 "up," the operator checks the third check point, adjusts the height and left / right of the fork 113, and confirms that the fork 113 can be inserted into the fork pocket of the object C based on this positional relationship. Note that the left / right adjustment of the fork 113 may also be performed during M13 "reach out." Furthermore, similar to flat loading, during M13 "reach out" and M14 "reach out," the operator confirms that the fork 113 does not interfere with the object C to be handled and is inserted all the way in during and after fork insertion.

[0061] When loading onto a shelf, the fourth check point photographed by the front camera 105 is the state of the object C, similar to the fourth check point when loading flat, but the condition of the object around this state includes, in particular, the clearance between the object C and the upper and lower beams of the shelf S after loading. Here, the clearance between the object C and the upper and lower beams of the shelf S refers to the vertical clearance between the top and bottom surfaces of the object C and the upper and lower beams until the object C is removed from the shelf S, and after the object C is removed from the shelf S, it refers to the clearance in the rear direction between the object C and the front ends of the upper and lower beams.

[0062] During M15 "Pick Up," the operator checks the status of the surrounding objects, specifically, that the vertical clearance allows the object C to be separated from the contact surface with the shelf S and not come into contact with the ceiling of the shelf S. If the shelf S is the top shelf of the rack R and does not have a ceiling, the operator checks that the object C does not come into contact with the ceiling of the building in which the rack R is located. During M17 "Reverse," the operator checks the status of the surrounding objects, specifically, that the rear clearance allows the object C to not come into contact with the front ends of the upper and lower beams. During M13 "Pick Up," M14 "Reach Out," M15 "Pick Up," M16 "Reach In," and M19 "Tilt," the operator checks the status of the object C itself, specifically, that the object C is stable.

[0063] When loading cargo onto the shelf, the fifth area to be checked, photographed by the rear camera 106, is the area behind the aircraft after loading. When M17 "reverse", the pilot checks that the area behind is safe.

[0064] When loading cargo onto a shelf, the sixth check point photographed by the front camera 105 is under the forks 113 when they are lowered after loading. When M18 "Down," the operator checks that there are no obstacles at the lowering position of the forks 113 loaded with the cargo object C, for example, that the front end of the cargo object C does not interfere with the front end of the rack or other cargo (obstacles including protruding parts or projections of cargo) stored in a rack below the loading height. In addition, the operator checks that the underside of the pallet of the cargo object C does not touch the top surface of the outriggers 111.

[0065] In this way, when loading cargo onto a shelf, the installation positions and shooting ranges of the front camera 105 and the rear camera 106 are determined so as to photograph the first to sixth confirmation points described above, namely, before loading, the direct facing relationship between the front of the machine body (backrest 121) and the cargo object C and / or the direct facing relationship between the front of the machine body (backrest 121) and the rack R, the part including the clearance between the tip of the machine body (outrigger 111) and the cargo object C and / or the part including the clearance between the tip of the machine body (outrigger 111) and the rack R, the positional relationship between the holding part (fork 113) and the holding part (fork pocket) of the cargo object C, the appearance of the cargo object C, and, after loading, the part including the clearance between the cargo object C and the upper and lower beams of the shelf S, the rear surroundings of the machine body, and the area under the forks when the raised forks 113 are lowered.

[0066] [During Unloading] Next, we will explain the areas to be checked during unloading, which are photographed by the front camera 105 or the rear camera 106. During unloading, a series of operations are being performed to unload the loading object C loaded by the forklift 101, and the operations differ slightly depending on whether the loading object C to be unloaded is placed flat or placed on a shelf S of the rack R, as will be described below.

[0067] <Operations during flat-bed unloading> When the forklift 101 unloads the loaded object C onto a flat surface (hereinafter referred to as flat-bed unloading), a series of specific operations are performed in the following order: M21 "Stop": The forklift 101 stops in front of the loading area Y. M22 "Reach out": The forklift 101 moves closer to the loading area Y from the stopped position. M23 "Reach out": The forklift 101 extends the forks 113 from the base 110 and moves the object C onto the loading area Y. M24 "Unload": The forklift 101 lowers the forks 113 and unloads the object C into the loading area Y. M25 "Reach in": The forklift 101 pulls the forks 113 toward the base 110. M26 "Reverse": The forklift 101 retreats to a position where it will not collide with the object C. After M26 "reverse", the forklift 101 can move forward or backward.

[0068] <Shelf-Loading Unloading Operation> The forklift 101 unloads the loaded object C onto the shelf S (loading location Y) of the rack R (hereinafter referred to as shelf-loading unloading). A series of specific operations are performed in the following order. Operations with the same name have the same content as flat-loading unloading. M31 "Stop": The forklift 101 stops in front of the rack R. M32 "Raise": The forklift 101 raises the forks 113, and raises the loaded object C to the height of the shelf S. M33 "Approach": The forklift 101 moves further toward the rack R from the stopped position. At this time, part of the object C is inserted into the shelf S. M34 "Reach Out": The forklift 101 extends the forks 113 from the base 110, and moves the entire object C into the shelf S. M35 "Unload": The forklift 101 lowers the forks 113 and drops the object C onto the shelf S. M36 "Reach in": The forklift 101 pulls the forks 113 towards the machine base 110. M37 "Reverse": The forklift 101 retreats to a position where it will not hit the rack R. M38 "Down": The forklift 101 lowers the forks 113, which are at the height of the shelf S. The height to which the forks 113 are lowered is a height where they do not touch the floor, for example, about 10 cm above the floor. After M38 "Down", the forklift 101 can move forward or backward.

[0069] As mentioned above, just like loading, when unloading, some operations are different between unloading on a flat surface and unloading on a shelf, so some check points are also different. Therefore, we will explain the check points separately for unloading on a flat surface and unloading on a shelf.

[0070] <Points to check when unloading flat> When unloading flat, the first point to be checked, which is photographed by the front camera 105, is the state and position of the loading area Y. Here, the state of the loading area Y refers to whether or not the loading object C can be safely unloaded at the loading area Y. Before / when M21 "stop" is executed, the driver checks that the loaded loading object C can be safely placed at the loading area Y. In addition, the driver grasps the position (front / back, left / right) of any loading object to be unloaded.

[0071] During flat-bed unloading, the second confirmation location photographed by the front camera 105 is the front of the vehicle, i.e., the direct facing relationship between the backrest 121 and the load placement area Y in the case of the forklift 101 of this embodiment. The direct facing relationship refers to a relationship in which the load placement area Y is located directly in front of the front of the vehicle, i.e., the backrest 121 in the case of the forklift 101 of this embodiment, and the backrest 121 and the load placement area Y are approximately parallel to each other. At the time of M21 "Stop," the operator confirms, based on this direct facing relationship, that the forklift 101 has stopped in a position where the load handling object C loaded at the load placement area Y can be unloaded. The fact that the load placement area Y is located directly in front of the backrest 121 can also be confirmed by checking that the load placement area Y is located between the outriggers 111a and 111b. Note that instead of the direct facing relationship between the backrest 121 and the load placement area Y, the front end of the load handling object C may be directly facing the load placement area Y.

[0072] The third check point photographed by the front camera 105 during flat unloading is the tip of the machine body, i.e., in the case of the forklift 101 of this embodiment, the part including the clearance between the tip of the outrigger 111 and the front edge of the loading area Y, or the part including the clearance between the innermost part of the loading area Y and the front edge of the object C, or both. During M22 "leaving," the operator uses this clearance to check that he is not getting too close to the loading area Y when leaving the loading area Y after stopping. The clearance may be any width, including zero.

[0073] During flat unloading, the fourth check point photographed by the front camera 105 is the state of the object C. Here, the state of the object C includes the state of the object C itself and the surrounding conditions of the object C, as in the case of flat loading. During M24 "unloading," the operator checks the surrounding conditions of the object C, specifically, checks that the object C being loaded has landed on the ground. Furthermore, during M23 "reach out," M24 ​​"unloading," and M25 "reach in," the operator checks the state of the object C itself, specifically, checks that the object C is stable.

[0074] The fifth check point, photographed by the front camera 105 during flat-bed unloading, is the positional relationship between the holding part and the object C, i.e., in this embodiment, the positional relationship between the fork 113 and the fork pocket of the object C, and confirms that the holding part holding the object C is in a position where it can be released. Before pulling out the fork during M25 "reach-in," the operator checks, based on this positional relationship, that the fork 113 is in a position where it can be pulled out from the fork pocket. Furthermore, during M25 "reach-in," the operator checks that the fork 113 is not interfering with the fork pocket of the object C while pulling out the fork, and after pulling out, the operator checks that the fork 113 has been completely pulled out from the fork pocket of the object C.

[0075] When unloading flat, the sixth area to be checked by the rear camera 106 is the area behind the aircraft after loading. When M26 "reverse", the pilot checks that the area behind is safe.

[0076] In this way, during flat-bed unloading operations, the installation positions and shooting ranges of the front camera 105 and the rear camera 106 are determined so as to photograph the first to sixth check points mentioned above, namely, the state of the loading area Y, the direct facing relationship between the front of the machine (backrest 121) and the loading area Y, the area including the clearance between the loading area Y and the front end of the machine (outriggers 111), the state of the object C to be loaded, the positional relationship between the holding part (fork 113) and the holding location (fork pocket) of the object C to be loaded, and the area around the rear of the machine.

[0077] <Shelf-mounted unloading> When unloading from a shelf, the first check point photographed by the front camera 105 is the same as the state of the loading location Y, which is the first check point when unloading flat. However, when unloading from a shelf, the loading location Y is shelf S of rack R, so before / when executing M21 "stop," the operator checks not only that the loading object C can be safely placed on shelf S, but also that the loading object C can be safely placed on rack R.

[0078] When unloading onto a shelf, the second check point photographed by the front camera 105 is the front of the machine body, which is the second check point when unloading flat, i.e., in the forklift 101 of this embodiment, the direct facing relationship between the backrest 121 and the object C and / or the direct facing relationship between the backrest 121 and the rack R. Note that, since the object C is placed on the shelf S of the rack R, the direct facing relationship between the backrest 121 and the rack R alone may be the second check point. Furthermore, when a line tape is drawn on the floor parallel to the front of the rack R, the direct facing relationship between the outrigger 111 and the line tape may be the second check point.

[0079] At the time of M31 "Stop," the operator confirms, based on the facing relationship, that the forklift 101 has stopped at a position where the object C loaded on the shelf S can be unloaded. The fact that the shelf S of the rack R is directly in front of the backrest 121 can also be confirmed by checking that the outriggers 111 fit within the width of the rack R. At the time of M11 "Stop," the operator also checks the clearance between the front end of the object C and the front end of the rack R, as photographed by the front camera 105, in addition to the first and second confirmation locations. This is because the front end of the object C needs to be in front of the front end of the rack R in order to raise the forks 113 at M32.

[0080] The third check point photographed by the front camera 105 during shelf unloading is the tip of the machine body, which is the third check point during flat unloading, i.e., in the case of the forklift 101 of this embodiment, the portion including the clearance between the tip of the outrigger 111 and the load placement location Y and / or the portion including the clearance between the rack R and the outrigger 111. Note that since the object C to be handled is placed on the shelf S of the rack R, only the portion including the clearance between the outrigger 111 and the rack R may be set as the third check point. Furthermore, if a line tape is drawn on the floor with a predetermined clearance from the rack R, the portion including the clearance between the outrigger 111 and the line tape may be set as the third check point.

[0081] When M33 "lifting", the operator checks that the clearance between the outriggers 111 and the rack R including the shelves S ensures that the outriggers 111 and the rack R, i.e., the outriggers 111 and the support frame and beams that make up the rack R, are not in contact with each other. The clearance may be any width, including zero.

[0082] The fourth check point photographed by the front camera 105 during shelf unloading is the appearance of the object C, similar to the fourth check point during flat-bed unloading. However, the surrounding conditions of the object C include, in particular, the clearance between the loaded object C and the upper and lower beams and left and right beams of the shelf S before unloading. Here, the clearance between the object C and the upper and lower beams and left and right beams of the shelf S refers to the vertical clearance between the loaded object C inserted / loaded on the shelf S and the upper and lower beams, and the horizontal clearance between the left and right beams. The clearance between the front end of the object C and the front end of the rack beam on which the shelf S is located is also checked to confirm that the entire object C fits within the shelf S.

[0083] During M33 "return," the operator checks the status of the surrounding cargo objects, specifically, that the loaded cargo objects C do not come into contact with the upper and lower beams and left and right beams of the shelf S. If the shelf S is the top shelf of the rack R and does not have a ceiling, the operator checks that the loaded cargo objects C do not come into contact with the ceiling of the building in which the rack R is placed. Note that during M33 "return," the operator simultaneously checks that the loaded cargo objects C do not come into contact with the upper and lower beams and left and right beams of the shelf S, and that the outriggers 111 do not come into contact with the support frame and beams that make up the rack R. In other words, during M33 "return," the operator checks the third and fourth check points simultaneously.

[0084] During M34 "reach out," the operator checks the status of the surrounding cargo objects, specifically, that the loaded cargo object C does not come into contact with the upper and lower beams and left and right beams of the shelf S. If the shelf S is the top shelf of the rack R and does not have a ceiling, the operator checks that the loaded cargo object C does not come into contact with the ceiling of the building in which the rack R is located. Furthermore, during M34 "reach out," the operator reaches out to a position where the front end of the cargo object C and the front end of the rack beam on which the shelf S is located no longer have clearance (they are aligned), and checks that the entire cargo object C is within the shelf S.

[0085] At the time of M35 "unloading," the operator checks the status of the surrounding objects, more specifically, that the loaded object C has touched the shelf S. At the time of M33 "loading," M34 "reach out," M35 "unloading," and M36 "reach in," the operator checks the status of the object C itself, more specifically, that the object C is stable.

[0086] During shelf unloading, the fifth check point photographed by the front camera 105 is the same as the fifth check point during flat-bed unloading: the positional relationship between the holding part and the object C, i.e., in this embodiment, the positional relationship between the fork 113 and the fork pocket of the object C, and checks that the holding part holding the object C is in a position where it can be released. During M36 "reach-in" and before pulling out the fork, the operator checks, based on this positional relationship, that the fork 113 is in a position where it can be pulled out from the fork pocket. During M36 "reach-in" and while pulling out the fork, the operator checks that the fork 113 is not interfering with the fork pocket of the object C, and after pulling out, the operator checks that the fork 113 has been completely pulled out from the fork pocket of the object C.

[0087] Next, when unloading cargo onto a shelf, the sixth check point photographed by the rear camera 106 is the area behind the aircraft after loading, similar to the sixth check point when unloading cargo flat. When M37 "reverse", the pilot checks for safety behind the aircraft.

[0088] When unloading goods from a shelf, the seventh check point photographed by the front camera 105 is the area under the forks 113 when the raised forks 113 are lowered after unloading. When M38 "lowering" is performed, the operator checks that there are no obstacles in the lowering position of the forks 113, and that the forks 113 are at a height that does not touch the floor, for example, about 10 cm above the floor.

[0089] In this way, when unloading an object C to be loaded onto a shelf S of a rack R, the installation positions and photographing ranges of the front camera 105 and the rear camera 106 are determined so as to photograph the first to seventh check points described above, namely, the state of the object C to be loaded, the state of the loading location Y before unloading, the direct facing relationship between the front of the machine body (backrest 121) and the object C to be loaded and / or the direct facing relationship between the front of the machine body (backrest 121) and the rack R, the part including the clearance between the tip of the machine body (outrigger 111) and the object C to be loaded and / or the part including the clearance between the tip of the machine body (outrigger 111) and the rack R, and the part including the clearance between the object C to be loaded and the upper and lower beams and left and right beams of the shelf S after loading, the positional relationship between the holding part (fork 113) and the holding part (fork pocket) of the object C to be loaded, and the rear surroundings of the machine body.

[0090] [During Travel] Finally, the following describes the areas to be checked while the forklift 101 is traveling, as photographed by the front camera 105 and the rear camera 106. Here, "while traveling" refers to an operating state in which the forklift 101 is moving forward or backward, and the forklift 101 may or may not be carrying an object C to be handled. The areas to be checked while traveling are the areas in front of and behind the forklift 101 and the direction of travel, including the travel path of the forklift 101. Note that the areas to be checked may also include the direction opposite to the direction of travel.

[0091] When traveling, the operator checks that there are no obstacles or people in front of and behind the vehicle and in the direction of travel, as well as the width of the travel path and the vehicle. Furthermore, when turning, the operator checks the turning point, and particularly when turning in an aisle where multiple racks R are installed, the operator checks that there is no interference between the vehicle and the racks R, or between the loaded objects C and the racks R. Furthermore, the operator may check the height from the floor of the holding parts and the objects C held by the holding parts, i.e., the forks 113 and the objects held by the forks 113 in this embodiment.

[0092] In this way, the installation positions and imaging ranges of the front camera 105 and the rear camera 106 are determined so that they capture images of the areas in front of and behind the forklift 101 and the direction of travel, including the travel path, when the forklift 101 is traveling. Note that, if the direction opposite to the direction of travel is also included in the areas to be checked, the installation positions and imaging ranges of the front camera 105 and the rear camera 106 are determined so that they capture images of the opposite direction.

[0093] <Functional Configuration> Figure 20 is a block diagram showing the functional configuration of the forklift remote operation system 10 according to this embodiment. The forklift remote operation system 10 is a system in which an operator at a remote location remotely controls the travel and loading of the forklift 101 at a work site using a remote control device 201. For example, in the forklift remote operation system 10, the operator operates the remote control device 201 to travel the forklift 101 at the work site and approach an object C to be handled, inserts the forks 113 into the fork pockets of a pallet P of the object C, and loads the object C. The operator then moves the loaded forklift 101 to a predetermined loading location Y and unloads the object C.

[0094] The remote control device 201 includes a communication unit 108-B that communicates wirelessly with the forklift 101, a control unit 202, and a display unit 203.

[0095] The operation unit 201 includes a handle, levers, pedals, etc., and allows the operator to operate the forklift 101 in the same manner as if the operator were actually riding on it and operating it, while viewing images displayed on a display unit 203 (described later). The operation unit 202 transmits operation commands input by the operator using the handle or the like to the forklift 101. The operation commands include forward, reverse, stop, reach out, reach in, mast forward / backward tilt (tilt), fork left / right, fork up / down, turn on the turn signal, press the horn, etc. The display unit 203 displays an image received from the forklift 101 via the communication unit 108-B. In this embodiment, the display unit 203 is described as being composed of two displays, but it may be composed of one or three or more displays.

[0096] The forklift 101 includes a drive system 102, a sensor 103, a camera 105, a vehicle control unit 104, an image processing unit 107, and a communication unit 108-A that communicates wirelessly with the remote control device 201.

[0097] The drive system 102 includes a traveling device (not shown) that travels in response to a traveling operation by the operator, and a loading / unloading device (not shown) that performs loading / unloading operations in response to a loading / unloading operation by the operator. The sensors 103 are sensors for acquiring information (also referred to as environmental information) related to the forklift 101 and the work site environment, such as a load presence sensor, a contact sensor, a distance sensor, an optical sensor, a temperature sensor, a three-dimensional sensor, a vibration sensor, and a weight sensor. The front camera 105 and the rear camera 106 photograph the areas to be checked by the operator when the forklift 101 is loading or traveling. In this embodiment, eight cameras are installed as shown in Figure 2, but the number of cameras is arbitrary. As described above, at least one camera, preferably at least one camera at the front and one at the rear, is sufficient as long as it can photograph the areas to be checked by the operator.

[0098] The vehicle control unit 104 acquires sensor information from the sensors 103. The vehicle control unit 104 also collects vehicle information, which is information related to each part of the forklift 101. The vehicle control unit 104 then controls the drive system 102, specifically the forks, wheels, mast, turn signals, etc., based on operation commands input to the steering unit 202. Here, the vehicle information includes pre-stored attribute information such as body dimension information of the forklift 101, and status information indicating the status of the forklift, such as vehicle speed, reach-out amount, mast angle, fork height, left and right fork position, steering angle, remaining battery level, remaining fuel level, and operating mode, which is acquired by the forklift's instruments (not shown) and sensors 103.

[0099] Image processing unit 107 generates a screen to be displayed on display unit 203 from images captured by front camera 105 and rear camera 106. The generated screen may be generated by randomly arranging images from each camera, or may be generated by arranging multiple images vertically or horizontally to provide a browsable view. Image processing for generating a screen by arranging multiple images to provide a browsable view will be described later with reference to FIG. 22 .

[0100] (Hardware Configuration) FIG. 21 is a diagram showing an example of the hardware configuration of the computer 900 provided in the forklift 101 and the remote control device 201 according to this embodiment.

[0101] The computer 900 includes a CPU 901, a memory 902, an auxiliary storage device 903, a communication interface (I / F) 904, an input / output interface (I / F) 905, and a GPU 906. The CPU 901, the memory 902, the auxiliary storage device 903, the communication interface (I / F) 904, the input / output interface (I / F) 905, and the GPU 906 are each connected to one another by a bus, which is a common transmission path.

[0102] The CPU 901 is a computing device that executes programs stored in the memory 902. The CPU 901 executes various programs to realize various functions of the device. Note that some of the processing performed by the CPU 901 by executing the programs may be executed by another computing device (hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)).

[0103] The memory 902 includes a non-volatile storage element, ROM (Read Only Memory), and a volatile storage element, RAM (Random Access Memory). The ROM stores unchanging programs (such as a Basic Input Output System (BIOS)). The RAM is a high-speed, volatile storage element, such as a Dynamic Random Access Memory (DRAM), and temporarily stores programs executed by the CPU 901 and data used during program execution.

[0104] The auxiliary storage device 903 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD: Hard Disk Drive) or a flash memory (SSD: Solid State Drive). The auxiliary storage device 903 also stores data used by the CPU 901 when executing a program, and the program executed by the CPU 901. That is, the program is read from the auxiliary storage device 903, loaded into the memory 902, and executed by the CPU 901 to realize each function of the device.

[0105] The communication I / F 904 is a network interface device that controls communication with other devices in accordance with a predetermined protocol. The input / output I / F 905 is an interface device to which devices that require input and output of electrical signals, such as the drive system device 102, the sensor 103, the cameras 105 and 106, the operation unit 202, and the display unit 203, are connected by wire or wirelessly, and which performs data acquisition, control, etc. using analog or digital electrical signals.

[0106] The GPU 906 is a processing unit specialized for image processing, and works in cooperation with the CPU 901 to execute some of the image processing at high speed.

[0107] 22 is a flowchart showing image processing performed by the image processing unit 107 in this embodiment to generate a screen by arranging multiple images so as to provide a viewable view. The screen generated by this flowchart provides a viewable view by arranging the images in accordance with the field of view from the cockpit.

[0108] The image processing unit 107 continuously and repeatedly executes this image processing at predetermined intervals according to the frame rate of the front camera 105 and / or the rear camera 106. In this flowchart, the image processing unit 107 starts processing when the power of the forklift 101 is turned on, and ends processing when the power of the forklift 101 is turned off. Note that the operator may be able to instruct the start / end of image processing regardless of the power of the forklift 101.

[0109] The image processing unit 107 acquires the traveling direction of the forklift 101 from the vehicle control unit 104 and determines whether the forklift 101 is traveling backward (S1001). If the forklift 101 is traveling backward, the process proceeds to S1002. On the other hand, if the forklift 101 is not traveling backward, i.e., if the forklift 101 is traveling forward or stopped, the process proceeds to S1004.

[0110] The image processing unit 107 generates a rear display screen B1 based on images captured by the rear cameras 106-1, 106-2, and 106-3 (hereinafter also referred to as image 106-1P, image 106-2P, and image 106-3P, respectively) (S1002).The image processing unit 107 generates a stopped front display screen F1 based on images captured by the front cameras 105-1, 105-2, 105-3, and 105-4 (hereinafter also referred to as image 105-1P, image 105-2P, image 105-3P, and image 105-4P, respectively) (S1003).

[0111] 23 shows examples of screen display and image layout of the rear display screen B1 and the front display screen F1 when the forklift 101 is stopped while reversing. The rear display screen B1 is generated by the image processing unit 107 by arranging the image 106-3P across the entire lower part of the screen and the images 106-1P and 106-2P in the upper right part of the screen.

[0112] The images 106-1P and 106-2P are arranged side by side in accordance with the left and right camera positions, and are displayed as an overhead image of the rear side of the base 110 of the forklift 101. Furthermore, when the forklift 101 turns, the images 106-1P and 106-2P are highlighted by the image processing unit 107, for example by surrounding them with a red frame. Such a highlighting method may be a flashing display in addition to a method of coloring the frame of the image. The image 106-3P is a wide-field-of-view image that includes the rear, front, and side, and is displayed as such when the forklift 101 is moving in reverse.

[0113] When the forklift 101 is traveling in reverse, it is important to check the direction of travel, so it is desirable that the image 106-3P be larger than the images 106-1P and 106-2P. As described above, the rear display screen B1 displays the rearward bird's-eye image based on the images 106-1P and 106-2P at the top right and the image 106-3P at the bottom, which is easy for the operator to understand visually, but the display is not limited to this, and the image may be upside down or the rearward bird's-eye image based on the images 106-1P and 106-2P may be in the center or on the left.

[0114] In this way, the rear display screen B1 provides a wide-view image in the direction of travel when reversing and a rearward bird's-eye image that includes the rear side of the forklift 101's base 110, allowing the operator to appropriately use the rear display image B1 to perform confirmation tasks when reversing, such as grasping the direction of travel when reversing and the surrounding area behind the forklift 101, and confirming safety. Also, by emphasizing only the image of the surrounding area behind the forklift 101, which is particularly important to check when turning, the operator can be urged to check.

[0115] The image processor 107 displays an image 105-1P on the left side of the screen, an image 105-2P on the right side of the screen, and an image 105-4P on the top center of the screen in the forward display screen F1 when the vehicle is stopped.１ Image 105-3P at the bottom center of the screen １ In the stationary front display screen F1, the image processing unit 107 processes the image 105-3P taken by the front camera 105-3 and the image 105-4P taken by the front camera 105-4, and generates an image 105-3P. １ , 105-4P １ In this processing, the image 105-4P is placed in the center of the screen. １ is image 105-3P １ This is because image 105-4P, which shows the area around the center of the front of the vehicle in the reverse direction, is more important than image 105-3P, which shows the area around the center of the front lower part of the vehicle, mainly including the fork 113. Processing when reversing and when stopped, which will be described later, will be described later.

[0116] In this way, the stopped front display screen F1 displays an image that allows the operator to see the entire area in front of the forklift 101, so that during remote control, the operator can see the field of view necessary for operation, and during on-board operation, the operator can see the area in front of the left side of the forklift 101 (image 105-1P) and the area around the forks 113 (image 105-3P), which are blind spots from the operator's seat 140. This allows the operator to properly check the area in front of the forklift 101 when reversing, whether during remote control or on-board operation.

[0117] 22, the image processing unit 107 generates a rear display screen B2 based on the images 106-1P, 106-2P, 106-3P, and 105-5P (S1004). Next, the image processing unit 107 acquires the traveling direction of the forklift 101 from the vehicle control unit 104 and determines whether the forklift 101 is traveling forward (S1005). If the forklift 101 is traveling forward, the process proceeds to S1006. On the other hand, if the forklift 101 is not traveling forward, i.e., if the forklift 101 is stopped, the process proceeds to S1007.

[0118] Then, the image processing unit 107 generates a forward display screen F3 for forward travel based on the images 105-1P, 105-2P, 105-3P, and 105-4P (S1006).

[0119] FIG. 24 shows examples of screen display and image layout of a rear display screen B2 when the forklift 101 is traveling forward and a front display screen F3 when traveling forward.

[0120] The rear display screen B2 is an image 106-3P, which is a reduced image size, displayed on the upper left of the screen by the image processing unit 107. １ The rearward bird's-eye view image based on the above-mentioned images 106-1P and 106-2P is generated by arranging the image 105-5P at the bottom right of the screen. １ The image processing unit 107 reduces the field of view of the image 106-3P to only the rear center front by deleting a predetermined width on both ends, thereby reducing the screen size. １ may simply be a reduced screen size of the image 106-3P.

[0121] The image 105-5P on the rear display screen B2 shows the front surroundings of the forklift 101 from the diagonally lower left to the right, and does not capture the areas to be checked during forward travel. Therefore, the images 106-1P, 106-2P, and 106-3P captured by the rear camera 106 are １ As described above, the rear display screen B2 is displayed with the image 106-3P at the top left of the screen. １ A rearward bird's-eye view image based on the above-mentioned images 106-1P and 106-2P is displayed at the top right of the screen, and image 105-5P is displayed at the bottom right of the screen, but this is not limiting, and the top, bottom, left, and right may be reversed, or image 105-5P may be in the center.

[0122] Thus, the rear display screen B2 has a smaller screen size and / or a smaller field of view for image 106-3P than the rear display screen B1, and therefore the operator's visibility is reduced, but because it provides a field of view image in the reverse direction of travel and a rearward bird's-eye view image including the rear side of the base 110 of the forklift 101 at the top of the screen, the operator can grasp the situation in the reverse direction of travel and the surroundings behind the forklift 101 and confirm safety even when the forklift 101 is traveling forward. Also, because the rear display screen B2 displays the surroundings in front of the forklift 101 from a diagonally downward left direction to the right, the operator can check the height of the forks 113 even when traveling forward.

[0123] The forward display screen F3 when moving forward is displayed by the image processing unit 107 as an image 105-1P at the bottom left of the screen. １ Image 105-2P appears in the bottom right corner of the screen. １ Image 105-4P is displayed at the top of the screen, and image 105-3P is displayed at the bottom center of the image. ２ are placed and generated.

[0124] On the forward display screen F3 when moving forward, the image processing unit 107 processes the images 105-1P, 105-2P, and 105-3P taken by the front cameras 105-1, 105-2, and 105-3, respectively, to display an image 105-4P that is taken widely in front of the vehicle and to the sides of the vehicle in a large size. １ , 105-2P １ , 105-3P ２ This allows the direction of travel to be displayed more widely, improving visibility. １ , 105-2P １ In the image 105-4P, the portions of the front wheels 112a and 112b and the road that are not included in the image 105-4P are cut out and displayed at the bottom left and bottom right of the screen, respectively. An example of processing during forward driving will be described with reference to FIG. 25, which will be described later.

[0125] Furthermore, when the forklift 101 is traveling forward, the forward display screen F3 is highlighted by the image processing unit 107 by surrounding the camera image 105-4P from the front camera 105-4 with a red frame.

[0126] An example of processing during forward travel will be described with reference to Fig. 25. When generating a forward travel front display screen F3, the image processing unit 107 first removes the upper portions from each of the images 105-1P and 105-2P and leaves the lower portions to generate an image 105-1P. １ , 105-2P １ is generated (S1101).

[0127] For example, the image processing unit 107 removes the upper part (upper 2 / 3 part) of a preset size from the images 105-1P and 105-2P to obtain the image 105-1P. １ , 105-2P １The generated image 105-1P １ , 105-2P １ includes at least a portion or all of the front wheel 112.

[0128] In addition, the image processing unit 107 deletes a predetermined lower region of the image 105-3P to obtain the image 105-3P. ２ After generating the image 105-3P, ２ is placed below the center of image 105-4P to generate intermediate image I1 (S1102). ２ For example, the generated image 105-3P may be generated by deleting a predetermined amount, such as the bottom quarter of the image 105-3P, or by deleting the part of the image vertically below that does not include the fork 113 or the part that only includes the floor as the bottom region. ２ is an image capturing the area below and in front of the forklift 101 including the outriggers 111 and the forks 113 when the forklift 101 is not loaded.

[0129] Then, the image processing unit 107 converts the intermediate image I1 into an image 105-3P １ Image 105-1 on the left and right Ｃ P, 105-2 Ｃ P is arranged to generate a forward moving forward display screen F3 (S1103). Note that S1101 and S1102 may be executed in any order, or may be executed simultaneously.

[0130] 22, the image processing unit 107 acquires the height of the forks 113 of the forklift 101 from the vehicle control unit 104 and determines whether the forks 113 are at or above a predetermined height (S1007). If the forks 113 are at or above the predetermined height, the process proceeds to step S1008. On the other hand, if the forks 113 are below the predetermined height, the process proceeds to step S1003.

[0131] 26 shows examples of screen display and image layout of the rear display screen B2 when the forklift 101 is handling cargo at a low altitude and the front display screen F1 when the forklift 101 is stopped. Note that the rear display screen B2 has been explained in FIG. 24, so explanation thereof will be omitted here.

[0132] Images 105-1P, 105-2P, and 105-3P on the front display screen F1 when the forklift 101 is stopped during low-place loading １ , 105-4 １ The arrangement and display of P are as described above, but the following highlighting is performed on the front display screen F1 when the vehicle is stopped, which was not performed when the vehicle was moving backward: １ When the forklift 101 approaches the object C and moves forward, the image processor 107 highlights the object C by surrounding it with a red frame.

[0133] In addition, the camera image 105-4P of the front camera 105-4 １ When the forks 113 are reaching out / in, the image is highlighted by being surrounded by a red frame by the image processing unit 107. In this way, by highlighting the image frame by coloring or the like based on the vehicle information of the forklift 101, the operator's line of sight can be guided to a screen that is appropriate for the work situation and environment, and confirmation and judgment can be supported.

[0134] Returning to FIG. 22, the image processing unit 107 synthesizes a stationary front display screen F2 based on the images 105-1P, 105-2P, 105-3P, and 105-4P (S1008).

[0135] 27 shows examples of screen display and image layout of the rear display screen B2 when the forklift 101 is handling cargo at a height and the front display screen F2 when the forklift 101 is stopped. Note that the rear display screen B2 has been explained in FIG. 24, so explanation thereof will be omitted here.

[0136] The image processor 107 displays the image 105-1P on the left side of the screen, the image 105-2P on the right side of the screen, and the image 105-4P on the top center of the screen in the forward display screen F2 when the vehicle is stopped. ３ Image 105-3P at the bottom center of the image ３As in the above-described stop front display screen F1, in the stop front display screen F2, the image processing unit 107 processes the image 105-3P and the image 105-4P to be placed in the center. Specifically, the image processing unit 107 arranges the image 105-3P, which shows the area around the front lower center including the clearance between the rack R and the outrigger 111, so as to be displayed enlarged. An example of the processing will be described with reference to FIG. 28, which will be described later.

[0137] In this way, the stopped front display screen F2, like the stopped front display screen F1, displays an image that allows the operator to see the entire area in front of the forklift 101, so that during remote operation the operator can see the field of view necessary for operation, and during on-board operation the operator can see the area in the front left (image 105-1P) and the area around the forks 113 (image 105-3P) that are blind spots from the operator's seat 140. Furthermore, the stopped front display screen F2 provides a large image of the area around the front lower center during high-altitude loading and unloading, so the operator can appropriately check the area in front, below, right, left, and right even during high-altitude loading and unloading.

[0138] As with the front display screen F1, the image 105-3P on the front display screen F2 is highlighted by the image processing unit 107 with a red frame when the forklift 101 is moving forward and approaching the object C. Also, the image captured by the front camera 105-4 is highlighted by the image processing unit 107 with a red frame when the forks 113 are reaching out / reach in.

[0139] An example of processing during stoppage will be described with reference to Fig. 28. First, the image processing unit 107 removes the left and right portions of the image 105-4P, leaving the center portion as the image 105-4P. １ For example, the image processing unit 107 vertically divides the image 105-4P into thirds, removes the left and right parts, and generates the image 105-4P by leaving the center part. １ It should be noted that the division into thirds does not have to be equal, and the image 105-4P １ It is desirable to divide the image 105-3P so that its width is the same as that of the image 105-3P.

[0140] Next, when generating the stop-time front display screen F1, the image processing unit 107 deletes the previously set lower area to generate the image 105-3P. １ After generating the image 105-3P, １ Image 105-4P １ The intermediate image I1 is generated by arranging the images 105-3P in parallel below the image 105-3P (S1202). １ is the image 105-3P mentioned above. ２ 105-3P is generated in the same manner as the image 105-3P, and is the same in that it is an image of the lower front of the forklift 101 including the outriggers 111 and the forks 113 in an unloaded state. １ and 105-3P ２ Since the size of the lower region to be deleted may differ between the two, they are indicated by different symbols.

[0141] Then, the image processing unit 107 places the image 105-1P on the left of the intermediate image I11 synthesized in S1202 and the image 105-2P on the right, thereby generating a front display screen F1 (S1203).

[0142] On the other hand, when synthesizing the stop-time front display screen F2, the image processing unit 107 synthesizes the image 105-4P generated in S1201. １ The image 105-3P is shifted vertically downward by a preset amount and superimposed on the image 105-3P to generate an intermediate image I12 (S1204). １ Image 105-4P, which is the upper region of ２ In the figure, as described above, the image 105-3P is arranged in parallel with the image 105-4P. １ It is superimposed directly on top of image 105-4P. １ removes the predefined lower area and creates image 105-4P. ２ After generating the image 105-3, the image 105-3 may be arranged in parallel on top of the image 105-3P to generate an intermediate image I2.

[0143] For example, image 105-4P １Image 105-3P is superimposed on the intermediate image I12, shifted vertically downward by 1 / 4. In this case, image 105-3P is entirely included in intermediate image I12, while image 105-4P １ The upper 1 / 4 part (Image 105-4P ２ ) is included in the intermediate image I12. As shown in FIG. 27, only the image 105-4P ２ In this case, the upper part of the object to be loaded onto the fork 113 needs to be included in the intermediate image I12 without overlapping with the image 105-3P, and the amount of shift is not necessarily the bottom 1 / 4 of the image vertically.

[0144] Then, the image processing unit 107 places the image 105-1P on the left of the intermediate image I12 generated in S1204 and the image 105-2P on the right of the screen, thereby generating a stopped front display screen F2 (S1205).

[0145] 22, the image processing unit 107 acquires the height of the forks 113 of the forklift 101 from the vehicle control unit 104 and determines whether the forks 113 are below a predetermined height (S1009). If the height is below the predetermined height, the process proceeds to step S1003. On the other hand, if the height is equal to or greater than the predetermined height, the process returns to step S1008. In this way, the image processing unit 107 acquires vehicle information about the forklift 101 and displays on the display unit 203 a screen that combines images (105-1P, 105-2P, 105-3P, 105-4P) captured by the front camera in a predetermined manner according to the height of the forks 113. In other words, the image displayed on the screen is switched, so that the operator can be provided with an image that includes areas to be checked during inspection work during travel and loading and unloading.

[0146] The image processing unit 107 determines whether the power of the forklift 101 has been turned off (S1010). If the power has been turned off, the process ends. On the other hand, if the power has not been turned off, the process returns to step S1001.

[0147] As described above, the forklift remote operation system of this embodiment can provide the operator with images including areas to be checked during checks during travel and loading / unloading. Furthermore, the system can improve visibility by enlarging and displaying images that require particular attention during travel and loading / unloading. It also highlights the image frame by coloring or flashing in accordance with environmental information about the work site and vehicle information about the forklift, guiding the operator's line of sight and calling attention, thereby supporting confirmation and judgment appropriate to the work situation and environment. This allows the operator to accurately grasp the situation and confirm safety using images, thereby ensuring the safety and accuracy of remotely operated forklift operations.

[0148] (Modification) In the above-described embodiment, the holding unit of the forklift 101 is two forks. However, the holding unit may be a pair of clamps, such as bale clamps or roll clamps. In this case, the third check point during flat loading and shelf loading, and the fifth check point during flat unloading and shelf unloading, are the positional relationship between the clamps (holding unit) and the side of the object C. In this way, the third check point during flat loading and shelf loading, and the fifth check point during flat unloading and shelf unloading, which check the positional relationship between the holding unit and the object C, check the positional relationship between the holding unit and the object C according to the holding method and holding location of the object C, which change depending on the shape of the holding unit. Note that check points other than those mentioned above during flat loading, shelf loading, flat unloading, and shelf unloading, i.e., check points other than those by the forklift's holding unit, are similar even for forklifts that use attachments other than two forks, such as bale clamps or roll clamps, as their holding units. Therefore, even for various forklifts with different shapes of holding parts, the forklift remote control system of this embodiment can provide the operator with images including the areas to be checked during checking work when traveling and loading and unloading.

[0149] REFERENCE SIGNS LIST 10 Remote control system 101 Forklift 102 Drive system device 103 Sensor 104 Vehicle control unit 105 Front camera 106 Rear camera 107 Image processing unit 108 Communication unit 201 Remote control device 202 Operation unit 203 Display unit

Claims

1. A forklift remote operation system comprising a forklift capable of communicating with each other and having a holding member for holding an object to be handled, and a remote operation device, wherein the forklift is equipped with one or more cameras whose installation position and imaging range are set so as to photograph, when the object to be handled, loaded on the holding member onto one shelf of a rack having a plurality of shelves arranged one above the other, before unloading: the state of the one shelf; the facing relationship between the front of the forklift body and the one shelf and / or the facing relationship between the front of the forklift body and the rack; the appearance of the loaded object to be handled, including the clearance between the one shelf and the front end of the forklift body and / or the clearance between the rack and the front end of the forklift body; the positional relationship between the holding member and the object to be handled; and, after unloading: the area around the rear of the forklift body and the area below the holding member as it descends; and the remote operation device A forklift remote control system equipped with a display unit that displays images captured by the one or more cameras on a screen.

2. A forklift remote control system as described in claim 1, wherein the installation position and imaging range of the one or more cameras are set so that, when the object to be handled stored on one shelf of the rack is loaded onto the holding member, before loading, the one or more cameras photograph the direct facing relationship between the front of the machine body and the object to be handled and / or the direct facing relationship between the front of the machine body and the rack, the portion including the clearance between the front end of the machine body and the object to be handled and / or the portion including the clearance between the front end of the machine body and the rack, and the positional relationship between the holding member and the object to be handled, and after loading, the installation position and imaging range are set so that the one or more cameras further photograph the appearance of the loaded object to be handled, including the clearance between the one shelf on both the top and bottom sides and the front end, the area around the rear of the machine body, and the area under the holding member as the holding member is lowered.

3. A forklift remote control system as described in claim 1, wherein the one or more cameras capture images of the surroundings of the vehicle and the traveling direction including the traveling path while the vehicle is traveling.

4. The forklift remote control system according to claim 1, wherein the one or more cameras include a camera installed near the base of at least one of the left and right outriggers of the forklift, and the camera's imaging range is the lower front area, including the outrigger on which the camera is not installed, when loading and unloading the object onto and from the one shelf.

5. A forklift remote control system as described in claim 1, wherein said one or more cameras include a pair of cameras installed on the left and right sides of a backrest that rises and falls in response to the extension and contraction of the mast of said forklift, said cameras having a wide range of imaging range in front of said forklift.

6. The forklift remote control system according to claim 1, wherein the one or more cameras include a camera installed in front of the forklift and a camera installed in rear of the forklift.

7. The forklift remote control system according to claim 1, wherein the one or more cameras include a forward center camera installed in the center of the machine body, and a forward lower center camera installed between the forks.

8. A remote control system as described in claim 7, wherein the display unit of the remote control device simultaneously displays the image captured by the front center camera on the screen, positioned above the image captured by the front lower center camera.

9. The remote control system according to claim 7, wherein the display unit of the remote control device acquires vehicle information about the forklift and switches images accordingly.

10. A remote control system as described in claim 9, wherein the display unit of the remote control device, when the forks of the forklift are below a predetermined height, generates a first front image by arranging the upper area of ​​the front lower center image taken by the front lower center camera in parallel below the front center image taken by the front center camera, making the upper area smaller than the front center image, and displays it on the screen.

11. A remote control system as described in claim 9, wherein the display unit of the remote control device, when the forks of the forklift are above a predetermined height, generates a second front image by arranging the upper area of ​​the front center image taken by the front lower center camera in parallel with the front lower center image taken by the front lower center camera, the upper area being smaller than the front lower center image, and displays the second front image on the screen.

12. A remote control system according to claim 9, wherein the display unit of the remote control device displays an image by enlarging or reducing the screen size of the image in accordance with the vehicle information of the forklift.

13. A remote control system as described in claim 7, wherein the display unit of the remote control device displays a front center image captured by the front center camera, and when the forklift moves forward, the screen size of the front center image is enlarged and displayed.

14. A forklift remote control system as described in claim 6, wherein the cameras installed at the rear include a pair of left and right rear cameras installed at the upper rear of the machine body, and a rear center camera installed at the center of the machine body.

15. A remote control system as described in claim 14, wherein the display unit of the remote control device displays a rear center image captured by the rear center camera, and when the forklift is moving backward, the screen size of the rear center image is enlarged and displayed.

16. The remote control system according to claim 1, wherein the remote control device highlights at least one image corresponding to vehicle information and / or environmental information of the forklift.

Images