Display device, visibility assistance system, industrial vehicle, control method and program

Abstract

To provide a display device which can check a range around a vehicle and of a dead angle by locating a virtual monitor at any position.SOLUTION: A display device, which is worn on the head of a person, comprises: a transparent display part that can display image data; an acquisition part that acquires an image photographed by a camera provided on an industrial vehicle; and a control part that displays the image data including a virtual monitor on which the image is displayed at a prescribed position of the industrial vehicle that can be seen through the display part in the display part.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a display device, a visibility assistance system, an industrial vehicle, a control method, and a program. [Background technology]

[0002] Patent Document 1 discloses a forklift operation support system that captures the surroundings of a vehicle using a camera mounted on a small unmanned aerial vehicle and displays the captured images on an on-board monitor attached in front of the driver's seat. Other systems include a system that captures the surroundings of a vehicle using a camera mounted on the vehicle and displays the images on a monitor attached in front of the driver's seat, and a system that indicates the direction of a detected person or object using sound or light. Even in systems that notify the detected direction using sound or light, when displaying an image of that direction, it is common for the image to be displayed on a monitor attached in front of the driver's seat. These systems allow the driver to identify people or obstacles in their blind spots by checking the monitor. However, if a monitor is attached to the driver's seat, the driver cannot check the monitor in front of the vehicle when, for example, backing up the vehicle while visually checking the area behind the vehicle. Furthermore, if a monitor is attached to the driver's seat, the monitor partially obstructs the driver's field of vision when driving the vehicle while facing forward. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-36102 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for technology that allows drivers to check the situation around the vehicle in the desired direction and blind spots without being restricted by the monitor's installation location.

[0005] The present disclosure provides a display device, a visibility assistance system, an industrial vehicle, a control method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0006] The display device of the present disclosure is a display device worn on a person's head, and includes a transparent display unit capable of displaying image data, an acquisition unit that acquires images captured by a camera installed on an industrial vehicle, a detection unit that detects people included in the images, and a virtual monitor on which the image is displayed, the image data being superimposed on a predetermined position on the industrial vehicle that is visible through the display unit, and when a person is detected by the detection unit, the display unit displays a message that a person has been detected, and when a person is not detected by the detection unit, By not displaying the person detection a control unit that displays on the display unit that no person has been detected When the specified position of the industrial vehicle moves out of the person's field of view, the control unit displays on the display unit that the person has been detected by displaying the direction in which the person was detected.

[0007] The visibility assistance system of the present disclosure includes a camera mounted on an industrial vehicle and the display device described above.

[0008] The industrial vehicle of the present disclosure includes a camera and the display device described above.

[0009] The control method of the present disclosure is a control method for a display device to be worn on a person's head, which has a transparent display unit capable of displaying image data, and acquires an image captured by a camera installed in an industrial vehicle. Steps to , detecting a person included in the image Steps to The image data including the virtual monitor on which the image is displayed is superimposed and displayed at a predetermined position on the industrial vehicle that can be seen through the display unit, and when a person included in the image is detected, the fact that a person has been detected is displayed on the display unit, and when a person included in the image is not detected, By not displaying the person detection The display unit displays a message that no person has been detected. and a display control step, in which, when a predetermined position of the industrial vehicle moves out of the field of view of the person, a display indicating the direction in which the person is detected is displayed on the display unit to indicate that the person has been detected.

[0010] The program disclosed herein also includes a computer for a display device to be worn on a person's head, the computer having a transparent display unit capable of displaying image data, a means for acquiring an image taken by a camera provided on an industrial vehicle, a means for detecting a person included in the image, and a virtual monitor on which the image is displayed, the image data being superimposed on a predetermined position on the industrial vehicle that is visible through the display unit, and when a person is detected by the detecting means, the display unit displays that a person has been detected, and when a person is not detected by the detecting means, By not displaying the person detection The display unit displays a message that no person has been detected. Display Control Function as a means When a predetermined position of the industrial vehicle moves out of the field of view of the person, the display control means displays a direction in which the person is detected, thereby displaying on the display unit that the person has been detected. . [Effects of the Invention]

[0011] According to the above-described display device, visibility assistance system, industrial vehicle, control method, and program, the surroundings of the vehicle and blind spots can be checked by placing a virtual monitor at any position. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram illustrating an example of a forklift according to an embodiment. [Figure 2] 1 is a block diagram illustrating an example of a display device according to an embodiment. [Figure 3A] FIG. 2 is a first diagram showing an example of the arrangement of virtual monitors according to the embodiment. [Figure 3B] FIG. 2 is a second diagram showing an example of the arrangement of virtual monitors according to the embodiment. [Figure 4A] FIG. 2 is a first diagram showing an example of visual information visible from the display device according to the embodiment. [Figure 4B] FIG. 2 is a second diagram showing an example of visual information visible from the display device according to the embodiment. [Figure 5] 10 is a flowchart illustrating an example of display control of a virtual monitor according to an embodiment. [Figure 6]FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] <Embodiment> Hereinafter, the display device of the present disclosure will be described with reference to the drawings. (composition) FIG. 1 is a schematic view showing an example of a forklift according to each embodiment. The forklift 1 includes a vehicle body 10, a loading device 11, and a display device 20. The vehicle body 10 includes front wheels 14, rear wheels 15, and a drive device and steering device (not shown), and drives the forklift 1. The loading device 11 includes forks 12 and a mast 13 for raising and lowering the forks 12, and performs loading and unloading operations by raising and lowering the load loaded on the forks 12. The vehicle body 10 is provided with one or more cameras 16. The cameras 16 are installed to capture areas that are likely to be blind spots for the driver 100, such as the rear and sides of the vehicle body 10. Images (which may be video or still images) captured by the cameras 16 are sent to the display device 20. The display device 20 is an AR (Augmented Reality) or MR (Mixed Reality) glass. The driver 100 wears the display device 20 to operate the forklift 1. In this specification, the direction in which the loading device 11 of the forklift 1 is provided as shown in FIG. 1 is referred to as the front, the opposite direction as the rear, and the left and right directions when facing forward are referred to as the left and right, respectively.

[0014] FIG. 2 is a block diagram showing an example of a display device according to an embodiment. The display device 20 is an AR or MR glass configured to include a display device and a computer (hereinafter, both will be referred to as AR glasses). The display device 20 is communicably connected to a control device and a camera 16 of the forklift 1 (not shown), and can display information obtained from these devices. For example, the display device 20 displays an image captured by the camera 16. As shown in the figure, the display device 20 includes a sensor unit 21, a display unit 22, a transmitter / receiver 23, a control unit 24, an obstacle detection unit 25, an audio output unit 26, and a storage unit 27.

[0015] The sensor unit 21 includes a sensor for head tracking of the driver 100 (for example, a plurality of visible light cameras and an IMU (Inertial Measurement Unit)), a sensor for eye tracking of the driver 100 (for example, an infrared camera), a depth sensor, a microphone, and the like.

[0016] The display unit 22 constitutes the lens (screen) of the AR glasses, and is configured, for example, with a transparent holographic display (the outside world can be seen through the lens). When no virtual object is projected onto the lens, the driver 100 can see the real world as it is. A virtual object is image data projected onto the display unit 22. When a virtual object is displayed on the display unit 22, the driver 100 sees the virtual object superimposed on the real world.

[0017] The transmission / reception unit 23 transmits and receives data to and from other devices. For example, the transmission / reception unit 23 transmits and receives data to and from other devices via wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). For example, the transmission / reception unit 23 receives images captured by the camera 16. The received images can be displayed on the display unit 22.

[0018] The control unit 24 performs display control of the display unit 22, etc. The control unit 24 recognizes the shape of the space around the driver 100 based on the information detected by the sensor unit 21, creates three-dimensional map information of the surrounding space, and generates a virtual space. The control unit 24 places a virtual object at a desired position in the virtual space based on the created map information. The control unit 24 calculates the position and posture of the driver 100 based on the information detected by the sensor unit 21, and if the virtual object placed in the virtual space is within the field of view of the driver 100, generates an image in which the virtual object is displayed at the placed position and projects the image onto the display unit 22. The virtual object is displayed to the driver 100 superimposed on an object that exists at that position in the real world. The virtual object can be displayed at (1) a predetermined position in space (the virtual object exists at a fixed position), (2) a predetermined position based on an object existing in the surrounding space (for example, if the object is the vehicle body 10, the virtual object moves to follow the movement of the vehicle body 10), or (3) a predetermined position based on the driver 100 (for example, the virtual object exists at a position in the direction the driver 100 is facing, regardless of the movement of the vehicle body 10). In the cases of (1) and (2), the virtual object is recognized by the driver 100 as if it actually exists at that position.

[0019] The above-described functions of the control unit 24 are functions provided in the AR glasses. The user sets the display device 20 for a desired virtual object (a developer uses an AR development tool to develop a virtual object and installs the developed program on the display device 20). The control unit 24 then displays the virtual object at a desired position according to the settings. In this embodiment, the control unit 24 is set to place a virtual monitor VM having a predetermined shape at a predetermined position on the vehicle body 10 and display an image captured by the camera 16 on the virtual monitor VM.

[0020] The virtual monitor VM is a virtual object, a virtual display screen that does not actually exist. The control unit 24 acquires an image captured by the camera 16 via the transmission / reception unit 23, and generates a virtual monitor VM on which the image is displayed. The control unit 24 places the virtual monitor VM at a predetermined position on the vehicle body 10 (corresponding to (2) above). The predetermined position on the vehicle body 10 is, for example, a predetermined position behind the vehicle body 10 (for example, any position within range h in FIG. 1 ). The virtual monitor VM is fixed at a predetermined position on the vehicle body 10, and therefore moves in accordance with the movement of the vehicle body 10. Since the virtual monitor VM is placed behind the vehicle body 10, for example, when the driver 100 is driving while looking ahead, the virtual monitor VM is not within the field of view of the driver 100 (it is not displayed on the display unit 22). Conversely, when the driver 100 looks back, the virtual monitor VM appears as if it were installed at a predetermined position behind the vehicle body 10. For example, control unit 24 calculates the field of view of driver 100 based on information detected by sensor unit 21, and if the installation position of virtual monitor VM is included in that field of view, generates image data in which virtual monitor VM is positioned. Control unit 24 calculates the installation position of virtual monitor VM on vehicle body 10 from the positional relationship between driver 100 and vehicle body 10, and displays virtual monitor VM at a position on display unit 22 that corresponds to the installation position of virtual monitor VM.

[0021] The control unit 24 also includes an obstacle detection unit 25. The obstacle detection unit 25 detects people and obstacles from images captured by the camera 16 by performing image processing such as pattern recognition. When the obstacle detection unit 25 detects a person or an obstacle, it notifies the control unit 24 of the position in the image where the person or obstacle is detected, the installation position of the camera 16 that captured the image, and the like. Based on the notification from the obstacle detection unit 25, the control unit 24 may, for example, highlight (for example, by displaying by surrounding with a conspicuous color) the area in which the person or obstacle is detected in the image displayed on the virtual monitor VM, or generate an image in which an arrow (the arrow is a virtual object) indicates the direction in which the person or obstacle is detected or the installation position of the camera 16 that captured the image in which the person or obstacle is detected.

[0022] The audio output unit 26 outputs sound. For example, the display device 20 is provided with a plurality of speakers, and can present a stereophonic sound to the driver 100. The storage unit 27 stores information detected by the sensor unit 21, information received by the transmission / reception unit 23, virtual objects, CAD information of the forklift 1 or various types of forklifts, and the like.

[0023] 3A and 3B show an example of the arrangement of virtual monitors VM. FIG. 3A is a side view of the vehicle body 10, and FIG. 3B is a top view of the vehicle body 10. As shown in FIGS. 3A and 3B, two cameras 16a and 16b are provided on the left and right sides of the rear of the vehicle body 10, respectively, and two virtual monitors VMa and VMb are disposed at predetermined positions behind the vehicle body 10 (for example, on top of a counterweight provided on the vehicle body 10). Due to the structure of the forklift 1, there is a blind spot behind the counterweight. Images of the blind spot in each direction are displayed on the virtual monitors VMa and VMb. For example, an image captured by camera 16a is displayed on the virtual monitor VMa, and an image captured by camera 16b is displayed on the virtual monitor VMb. By arranging cameras 16a, 16b and virtual monitors VMa, VMb in this manner and displaying images captured by cameras 16a, 16b on virtual monitors VMa, VMb, for example, when driver 100 looks back and reverses forklift 1, he or she can drive while checking areas that are likely to be blind spots on virtual monitors VMa, VMb.

[0024] The installation positions, shooting directions, and number of cameras 16a, etc. illustrated in FIGS. 3A and 3B are merely examples. For example, cameras 16 may be provided on both side surfaces of the vehicle body 10, or may capture images of the rear or diagonally rearward direction from the side of the vehicle body 10. The placement positions, number, size, and shape of the virtual monitors VMa and VMb illustrated in FIGS. 3A and 3B are merely examples. For example, virtual monitors VM may be provided in the same number as the cameras 16, and images captured by each camera 16 may be displayed on each virtual monitor VM. Alternatively, only one virtual monitor VM may be provided, and the display unit of the virtual monitor VM may be divided so that images captured by each camera 16 are displayed in each divided area. Alternatively, images captured by one camera 16 may be divided based on the shooting direction and displayed on two virtual monitors VMa and VMb. For example, one camera 16 may be installed in the center of the rear of the vehicle body 10, the captured image may be divided into left and right halves, and the image in the direction of the virtual monitor VMa may be displayed on the virtual monitor VMa, and the image in the direction of the virtual monitor VMb may be displayed on the virtual monitor VMb.

[0025] FIG. 4A shows an example of visual information that the driver 100 sees through the display device 20 when he or she looks back. As shown in the figure, the driver 100 sees as if two virtual monitors VMa and VMb, each displaying an image captured by the camera 16, are installed at the rear of the vehicle body 10. In this example, the virtual monitors VMa and VMb are positioned within the housing of the vehicle body 10 visible to the driver 100, allowing the driver 100 to view the image captured by the camera 16 without obstructing the driver's view of the real world. The triangular notification display 41 is a virtual object that indicates the direction in which a person or other object is detected by the obstacle detection unit 25. More specifically, it indicates the direction of the location of the virtual monitor VM displaying the image in which a person or other object is detected, or the direction of the installation location of the camera 16 that captured the image in which a person or other object is detected. The display position, size, shape, etc. of the triangular notification display 41 are not limited to those shown in the figure. For example, the position where a person or the like is detected may be analyzed in more detail, and a virtual object that more accurately indicates the detected position may be displayed, or text information such as "left" or "right" may be displayed. Furthermore, notification display 41 may be displayed not only when driver 100 turns around, but also when the driver is facing forward while driving. This is shown in FIG. 4B (in FIG. 4B, the outside world view visible through display unit 22 is omitted). If no person or obstacle is detected, control unit 24 does not display triangular notification display 41.

[0026] The highlight 42 in FIG. 4A is a virtual object for highlighting the area in which a person or other object is detected when the image displayed on the virtual monitor VM includes a person or other object (the image shown does not include a person or other object, and the highlight 42 in FIG. 4 is for illustrative purposes only). The highlight 43 in FIG. 4A is a virtual object for highlighting the virtual monitor VM when the image displayed on the virtual monitor VM includes a person or other object. When the obstacle detection unit 25 detects a person or other object, the control unit 24 generates the highlight 42 indicating the position and area in which the person or other object is detected, and displays the image data of the person or other object on the display unit 22 at a position corresponding to the arrangement position of the virtual monitor VM. The color and highlighting method of the highlight 42 are not limited to those shown in the figure. When no person or obstacle is detected in the image captured by the camera 16, the control unit 24 does not perform the highlighting. The same applies to the highlight 43.

[0027] (operation) Next, control of the display device 20 will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of display control of the virtual monitor according to the embodiment. When the driver 100 wearing the display device 20 sits in the driver's seat of the forklift 1, the driver starts up the display device 20 and performs calibration (step S1). In the calibration, the positional relationship between the driver 100 and the forklift 1 is calculated. For example, the control unit 24 recognizes the handle and other structures of the forklift 1 using the visible light camera of the sensor unit 21, and estimates the type of vehicle of the forklift 1 from the shape and appearance of the handle, etc. that are photographed. The control unit 24 reads out CAD information corresponding to the estimated type of vehicle from the CAD information recorded in the memory unit 27, and grasps the overall shape of the forklift 1. Furthermore, for example, the control unit 24 may analyze an image of the steering wheel captured by the visible light camera of the sensor unit 21, or measure the distance to the steering wheel using the depth sensor of the sensor unit 21, thereby calculating the positional relationship between the driver 100 and the steering wheel, i.e., where the driver 100 is sitting in the driver's seat, how high the driver's 100's head is, or, from the read CAD information of the forklift 1, the positional relationship (horizontal distance, vertical distance, etc.) between the driver 100 and each part of the forklift 1. This makes it possible to place the virtual monitor VM in a desired position. Alternatively, for example, a two-dimensional marker may be provided at a predetermined position on the forklift 1, the sensor unit 21 may recognize the two-dimensional marker, and the positional relationship between the driver 100 and each part of the forklift 1 may be calculated in the same manner as above, based on the positional relationship with the two-dimensional marker, the model of the forklift 1 indicated by the two-dimensional marker, and the CAD information of the model.

[0028] Once the calibration is complete, the driver 100 performs a predetermined operation on the display device 20 to activate a function for displaying the virtual monitor VM. When this function is activated, the control unit 24 places the virtual monitor VM at a predetermined position in the virtual space (for example, behind the vehicle body 10). The obstacle detection unit 25 acquires an image captured by the camera 16 via the transmission / reception unit 23. The obstacle detection unit 25 analyzes the acquired image to detect people and obstacles (step S2). If a person or other object is detected (step S3; Yes), the obstacle detection unit 25 outputs the position and direction in which the person or other object was detected and the image captured by the camera 16 to the control unit 24. The control unit 24 generates and arranges a highlight 42 based on information about the position in which the person or other object was detected, and generates and arranges a notification display 41 and a highlight 43 based on information about the direction in which the person or other object was detected (step S4). For example, the control unit 24 arranges the notification display 41 at the vertical center of the field of view and at the horizontal end in the direction in which the person or other object was detected. The control unit 24 may execute all of the generation and placement of the notification display 41 and the highlight displays 42 and 43, or may execute any one or two of them. Next, the control unit 24 displays the image captured by the camera 16 on the virtual monitor VM (step S5).

[0029] If no person or the like is detected (step S3; No), obstacle detection unit 25 outputs information indicating that no person or the like is detected and the image captured by camera 16 to control unit 24. Control unit 24 displays the image captured by camera 16 on virtual monitor VM (step S5).

[0030] The control unit 24 controls the display of the virtual monitor based on the driver's posture (step S6). For example, the control unit 24 calculates the field of view of the driver 100 based on the information detected by the sensor unit 21, and displays the virtual monitor VM if the field of view includes the placement position of the virtual monitor VM. At this time, the direction of the virtual monitor VM may be adjusted according to the head position and posture of the driver 100. For example, if the line of sight of the driver 100 (the line of sight is an example of the head position) is high, the upward angle of the display surface of the virtual monitor VM is increased to match the line of sight of the driver 100, and if the line of sight of the driver 100 is low, the upward angle of the display surface of the virtual monitor VM is decreased. For example, when the driver 100 turns his / her body leftward (the direction of turning the body is an example of the posture of the driver 100) and looks back, the display surface of the virtual monitor VMb (the display surface normally faces forward) disposed at the rear right of the vehicle body 10 is turned to face leftward and forward in line with the line of sight of the driver 100. Furthermore, the control unit 24 does not display the virtual monitor VM if the position of the virtual monitor VM is not included in the field of view of the driver 100, and displays the notification display 41 if the notification display 41 has been generated. When the driver 100 is facing forward, the virtual monitor VM is not displayed, and if a person or an object is detected, only the notification display 41 indicating the direction in which the person or an object is detected is displayed. When the driver 100 looks back, the virtual monitor VM is displayed, and if a person or an object is detected, at least one of the notification display 41 and the highlight display 42 or 43 is displayed. This allows the driver to drive without obstructing his / her field of view when facing forward, and allows the driver to know the direction when a person or an object is detected behind the vehicle. In a situation where the driver 100 needs to look back to back up the forklift 1, the virtual monitor VM is displayed, allowing the driver to drive while checking the blind spot on the virtual monitor VM. Next, the control unit 24 determines whether to terminate the function of displaying the virtual monitor VM (step S7). For example, if the driver 100 performs an operation to terminate the function of displaying the virtual monitor VM, the control unit 24 determines that the function of displaying the virtual monitor VM should be terminated; otherwise, the control unit 24 determines that the function should not be terminated.If it is determined not to end (step S7; No), the processes from step S2 onward are repeated. If it is determined to end (step S7; Yes), the process flow of FIG. 5 ends.

[0031] In the above embodiment, when a person or the like is detected, a visual notification is given, but the display device 20 may notify the direction in which the person is located by outputting a sound from the audio output unit 26. Furthermore, in the above embodiment, an example has been described in which the visibility assistance system including the camera 16 and the display device 20 is applied to the forklift 1, but the visibility assistance system can also be applied to industrial vehicles other than forklifts, such as an in-plant transporter, an in-plant towing vehicle, an excavator, a bulldozer, a crane, and a reach stacker.

[0032] (effect) Conventionally, systems have been provided in which a real monitor is installed in front of the driver's seat and an image captured in a blind spot is displayed on that monitor. With this system, it is not possible to check the monitor in front of the vehicle while visually checking the area behind the vehicle. Furthermore, even if a person or obstacle is detected in the vehicle's blind spot, the driver must look at the monitor in front of the vehicle to confirm its presence on video. Furthermore, the monitor in front of the driver's seat obstructs the driver's field of view when it is not necessary. In contrast, in this embodiment, the driver wears AR glasses (display device 20), and a virtual monitor VM is displayed in a virtual space, and an image of the area around the vehicle captured by camera 16 is displayed on the virtual monitor VM. This allows the driver 100 to grasp the situation around the vehicle by checking the virtual monitor VM even when looking back. Furthermore, the driver's field of view is not obstructed when the driver 100 is facing forward.

[0033] FIG. 6 is a diagram illustrating an example of a hardware configuration of the display device according to the embodiment. The computer 900 includes a CPU 901 , a main memory device 902 , an auxiliary memory device 903 , an input / output interface 904 , and a communication interface 905 . The above-described display device 20 is implemented in a computer 900. Each of the above-described functions is stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903, loads it into the main storage device 902, and executes the above-described processing in accordance with the program. The CPU 901 also allocates a storage area in the main storage device 902 in accordance with the program. The CPU 901 also allocates a storage area in the auxiliary storage device 903 for storing data being processed in accordance with the program.

[0034] Alternatively, a program for implementing all or part of the functions of the display device 20 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform processing by each functional unit. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, if a WWW system is used, the term "computer system" also includes a homepage provision environment (or display environment). Furthermore, the term "computer-readable recording medium" refers to portable media such as CDs, DVDs, and USBs, as well as storage devices such as hard disks built into the computer system. Furthermore, if the program is distributed to the computer 900 via a communication line, the computer 900 that receives the program may load the program into the main storage device 902 and execute the above-described processing. Furthermore, the program may be for implementing part of the above-described functions, or may be capable of implementing the above-described functions in combination with a program already stored in the computer system.

[0035] As described above, several embodiments according to the present disclosure have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as defined in the claims, as well as in the scope and spirit of the invention.

[0036] <Additional Notes> The display device, the visibility assist system, the industrial vehicle, the control method, and the program described in each embodiment can be understood, for example, as follows.

[0037] (1) The display device according to the first aspect is a display device worn on the head of a person (driver 100 or passenger), and comprises a transparent display unit 22 capable of displaying image data, an acquisition unit (transmitter / receiver 23) that acquires images captured by a camera 16 provided on an industrial vehicle (forklift 1), and a control unit 24 that displays the image data, including a virtual monitor VM on which the image is displayed, at a predetermined position on the industrial vehicle on the display unit 22 that is visible through the display unit. This means that by using AR technology to install a virtual monitor VM at any position on an industrial vehicle, it will be possible, for example, to check images of blind spots behind the vehicle while looking behind it.In addition, it is possible to remove the virtual monitor VM itself in situations where it is not necessary (such as when looking forward), thereby improving forward visibility.

[0038] (2) A display device according to a second aspect is the display device of (1), wherein the control unit adjusts the orientation of the virtual monitor according to the positional relationship between the head and the virtual monitor. This makes it easier to check the virtual monitor.

[0039] (3) A display device according to a third aspect is a display device according to (1) to (2), further comprising a detection unit (obstacle detection unit 25) that detects a person included in the image, and when a person is detected by the detection unit, the control unit displays on the display unit that a person has been detected. This allows the presence of a person around the vehicle to be detected.

[0040] (4) A display device according to a fourth aspect is the display device of (3), in which the control unit displays on the display unit that the person has been detected by highlighting the virtual monitor. This allows the driver to be notified that a person is present around the vehicle.

[0041] (5) A display device according to a fifth aspect is a display device according to (3) to (4), wherein the control unit displays on the display unit that the person has been detected by highlighting the person included in the virtual monitor. This allows the driver to be notified that a person is present around the vehicle.

[0042] (6) A sixth aspect of the display device is a display device according to any one of (3) to (5), wherein when a predetermined position of the industrial vehicle moves out of the field of view of the person, the control unit displays on the display unit that the person has been detected by displaying a display indicating the direction in which the person was detected. This allows the driver to be notified that a person is present around the vehicle.

[0043] (7) A display device according to a seventh aspect is a display device according to any one of (3) to (6), further comprising an audio output unit that outputs sound, and when a person is detected by the detection unit, the audio output unit outputs a sound indicating that a person has been detected. This allows the driver to be notified that a person is present around the vehicle.

[0044] (8) A visibility assistance system according to an eighth aspect includes a camera mounted on an industrial vehicle and a display device according to any one of (1) to (7).

[0045] (9) An industrial vehicle according to a ninth aspect includes a camera and a display device according to any one of (1) to (7).

[0046] (10) A control method according to a tenth aspect is a control method for a display device worn on a person's head, which has a transparent display unit capable of displaying image data, and which acquires an image captured by a camera installed on an industrial vehicle, and displays the image data, including a virtual monitor on which the image is displayed, at a predetermined position on the display unit of the industrial vehicle that can be seen through the display unit.

[0047] (11) The program according to the eleventh aspect causes a computer of a display device worn on a person's head, which has a transparent display unit capable of displaying image data, to function as a means for acquiring images captured by a camera installed on an industrial vehicle, and a means for displaying the image data, including a virtual monitor on which the image is displayed, at a predetermined position on the display unit of the industrial vehicle that can be seen through the display unit. [Explanation of symbols]

[0048] 1. Forklift 10. Body 11. Loading equipment 12...fork 13 Mast 14. Front wheel 15 Rear wheel 16. Camera 20...Display device 21 Sensor section 22...Display section 23 Transmitter / receiver 24 Control section 25 Obstacle detection unit 26 Audio output section 27...Storage section 41...Notification display 42...highlight 100···Driver VM, VMa, VMb...Virtual monitors 900···Computer 901 CPU 902...Main memory 903...Auxiliary storage device 904 Input / Output Interface 905···Communication Interface

Claims

1. A display device worn on a person's head, a transparent display unit capable of displaying image data; an acquisition unit that acquires images captured by a camera installed in the industrial vehicle; a detection unit that detects a person included in the image; a control unit that displays the image data including a virtual monitor on which the image is displayed, superimposed on a predetermined position of the industrial vehicle that is visible through the display unit, and that, when a person is detected by the detection unit, displays on the display unit that a person has been detected, and, when a person is not detected by the detection unit, displays on the display unit that a person has not been detected by not displaying information regarding the detection of a person; Equipped with When a predetermined position of the industrial vehicle moves out of the field of view of the person, the control unit displays a display indicating the direction in which the person is detected, thereby displaying on the display unit that the person has been detected. Display device.

2. the control unit adjusts the orientation of the virtual monitor according to the positional relationship between the head and the virtual monitor. The display device according to claim 1 .

3. The acquisition unit acquires the image captured by the camera that captures an image of the side or rear of the industrial vehicle, the control unit displays the image data including the virtual monitor at a position behind the vehicle body of the industrial vehicle that can be seen when the driver turns around. The display device according to claim 1 or 2.

4. the control unit displays the virtual monitor in an emphasized manner to indicate on the display unit that the person has been detected. The display device according to claim 1 or 2.

5. the control unit displays the person included in the virtual monitor in an emphasized manner, thereby displaying on the display unit that the person has been detected. The display device according to claim 1 or 2.

6. An audio output unit that outputs sound, the audio output unit, when a person is detected by the detection unit, outputs a sound indicating that a person has been detected. The display device according to claim 1 or 2.

7. A camera installed in an industrial vehicle; The display device according to claim 1 or 2; A visual aid system.

8. Camera and The display device according to claim 1 or 2; An industrial vehicle equipped with:

9. A method for controlling a display device worn on a person's head, the display device having a transparent display unit capable of displaying image data, comprising: acquiring an image captured by a camera installed in the industrial vehicle; detecting a person in the image; a display control step of superimposing and displaying the image data including a virtual monitor on which the image is displayed at a predetermined position on the industrial vehicle that is visible through the display unit, and when a person is detected in the person detection step, displaying on the display unit that a person has been detected, and when a person is not detected in the person detection step, displaying on the display unit that a person has not been detected by not displaying information regarding the person detection; and In the display control step, when a predetermined position of the industrial vehicle moves out of a visual field of the person, a display indicating a direction in which the person is detected is displayed on the display unit, thereby displaying that the person has been detected. Control method.

10. A computer for a display device worn on a person's head, the computer having a transparent display capable of displaying image data, a means for acquiring images captured by a camera installed in the industrial vehicle; means for detecting a person included in the image; a display control means for displaying the image data including a virtual monitor on which the image is displayed, superimposed at a predetermined position on the industrial vehicle that is visible through the display unit, and for displaying on the display unit, when a person is detected by the detecting means, a message indicating that a person has been detected, and, when a person is not detected by the detecting means, a message indicating that a person has not been detected by not displaying any message regarding the detection of a person; It functions as When a predetermined position of the industrial vehicle moves out of the field of view of the person, the display control means displays a direction in which the person is detected, thereby notifying the display unit that the person has been detected. program.

Images