Video display device and ar glass

Abstract

To provide AR glasses with a wide field of view and easy-to-read information display.SOLUTION: AR glasses are provided that mount a video display device having an imaging optical system that forms an image of the subject, an imaging element that converts the image of the subject formed by the imaging optical system into electrical signals, a video processing unit that converts electrical signals from the imaging element into video signals, a display unit for displaying video signals converted by the video processing unit, and eyepiece lens for observing images displayed on the display unit at actual size, over one eye while allowing the other eye to see through transparently.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a video display device that provides a field of view that enables augmented reality, and to AR glasses. [Background technology]

[0002] In recent years, AR (Augmented Reality) glasses have become increasingly popular as information and communication devices. There are two types of AR glasses: optically transparent and video transparent, both of which have their advantages and disadvantages.

[0003] An example of the optical transmission type is disclosed in Patent Document 1, in which an image is projected onto a half-mirror screen and information is superimposed on the actual field of view to realize augmented reality.

[0004] On the other hand, an example of a video transmission type is disclosed in Patent Document 2, in which an image captured by a camera is displayed on an HMD (Head Mount Display) that covers the entire field of view, and augmented reality is realized by superimposing various information on the image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-51230 [Patent Document 2] Patent No. 5329480 Summary of the Invention [Problem to be solved by the invention]

[0006] Optically transparent AR glasses have a lower visibility than when not wearing the glasses, due to factors such as the half mirror darkening the real field of view and the image projection mechanism obscuring parts of the field of view. They are also susceptible to environmental influences, which can make projected images difficult to see.

[0007] While video-transparent AR glasses can provide realistic images, they can be unsettling when walking around because the user is not viewing the real field of view. Rapid movement can also lead to motion sickness. Furthermore, if a malfunction occurs, such as a power outage, the user loses all vision, posing a safety issue.

[0008] The present invention was devised to address the above-mentioned issues, and provides a safe and convenient augmented reality field of view by solving the problems of both optical transmission and video transmission types. Furthermore, the present invention provides AR glasses that allow users to view various images outside the field of view with high image quality. [Means for solving the problem]

[0009] In order to solve the above problem, a first aspect of the image display device of the present invention comprises a first photographing optical system that forms an image of a subject, a first imaging element that converts the subject image formed by the first photographing optical system into an electrical signal, a first image processing unit that converts the electrical signal from the first imaging element into a video signal, a display unit that displays the video signal converted by the image processing unit, and an eyepiece for observing the image displayed on the display unit by magnifying it to the same size as the field of view, thereby allowing the user to view an image of the same size as the field of view.

[0010] A second aspect of the video display device of the present invention, in addition to the first aspect, further includes a communication unit that acquires information from an external device via wireless communication, and the first video processing unit superimposes the information acquired by the communication unit on the converted video signal and displays it on the display unit.

[0011] A third aspect of the image display device of the present invention is the same as the first aspect, except that it further comprises: a second photographing optical system that forms an image of a subject; The image display device has a second image sensor that converts the subject image formed by the second photographing optical system into an electrical signal, a second image processing unit that converts the electrical signal from the second image sensor into a video signal, and a display switching unit that switches between the image from the first image processing unit and the image from the second image processing unit based on instructions from the user and outputs the switched image to the display unit.

[0012] In a fourth aspect of the video display device of the present invention, the communication unit acquires video information from an external device, and the first video processing unit converts the video information acquired from the communication unit into a video signal that can be displayed on the display unit and displays it on the display unit.

[0013] The AR glasses according to a fifth aspect of the present invention include a frame for fixing the image display device according to any one of the first to fourth aspects to one of the eyes. [Effects of the Invention]

[0014] We provide AR glasses that provide an augmented reality field of view with improved visibility. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a functional block diagram showing the functional configuration of a video display device V according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing an example of a state in which the AR glasses G of the first embodiment are worn; [Figure 3] FIG. 1 is a side view of the AR glasses G according to the first embodiment. [Figure 4] FIG. 1 is a diagram for explaining the field of view obtained with the AR glasses G of the first embodiment. [Figure 5] FIG. 10 is a diagram showing the relationship between the imaging angle of view and the observation angle of view of the image display device V of the first embodiment. [Figure 6] A functional block diagram showing the configuration of a video display device V according to a second embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of the appearance of AR glasses according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] <First embodiment: Overview> The image display device V of the present invention is attached to the eyeglass frame GF of the AR glasses G in place of one of the lenses, as shown in Fig. 2. The image display device V is configured so that one eye views an object through the image displayed by the image display device V, while the other eye views the object directly. When the size of the image on the image display device V is the same as the size seen in the field of view, a synthetic field of view that does not feel strange can be provided. Here, the synthetic field of view means a field of view that is a combination of the real field of view seen directly and the image field of view seen through the image. Furthermore, by superimposing information on the image of the image display device V, the user can see the information as if it were being displayed within their field of vision, and so the device can be used as AR glasses that realize augmented reality.

[0017] <First embodiment: configuration> FIG. 1 is a functional block diagram showing the functional configuration of a video display device V according to the first embodiment. The image display device V is composed of an imaging optical system 11, an image pickup element 12, an image processing unit 2, a display unit 32, an objective lens 31, an eyecup 33, and a communication unit 4.

[0018] First Embodiment: Description of Configuration: Photographic Optical System The photographing optical system 11 forms an image of a light beam from the user's field of view on the imaging surface of the imaging element 12 . The photographic optical system 11 has a single focal length, and it is preferable that the focal length be short so as to ensure a wide photographic angle. Also, it is preferable that the photographic optical system 11 has a deep depth of field, because this makes it more likely that the entire field of view will be in focus.

[0019] There are no particular restrictions on the projection method of the photographic optical system, but a fisheye optical system is suitable because it has a wide angle of view. Furthermore, among fisheye optical systems, equidistant projection is preferable because the calculation required for conversion to other projection methods through image processing is relatively easy in the case of equidistant projection.

[0020] First Embodiment: Description of Configuration: Image Sensor The image sensor 12 photoelectrically converts the light beam from the image formed on the image pickup surface for each pixel and outputs an electrical signal. The imaging element 12 is, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The higher the frame rate of the image sensor 12, the better, as the image will be less likely to be distorted even if the user moves vigorously. For example, 60 fps (frames per second) or higher is desirable, and 120 fps is sufficient.

[0021] <First embodiment: Description of configuration: video processing unit> The image processor 2 converts the electrical signal read from the image sensor 12 into an image signal in a format that can be displayed on the display unit 32 and outputs the image signal. The electrical signal read from the imaging element 12 is output in synchronization with a horizontal synchronization signal and a vertical synchronization signal, and the electrical signal is associated with coordinate information and converted into a video signal. Furthermore, the video processing unit 2 can perform various video processes on the video signal. For example, various methods are conceivable, such as trimming the video signal to a range of a predetermined angle of view, adjusting the brightness of the video signal, or enhancing the contours of the video signal.

[0022] When the video processing unit 2 receives information data to be displayed from the communication unit 4, it converts the information data to be displayed into an image, superimposes it at a predetermined coordinate position on the video signal, and outputs it. The information data includes, for example, the display position (coordinates), the text or image to be displayed, and information such as color and size.

[0023] Furthermore, the video processing unit 2 can also acquire video data from the communication unit 4. When the user selects to display video from an external device, the video processing unit 2 stops outputting the video signal from the image sensor 12 and converts the video data acquired from the communication unit 4 into a video signal that can be displayed on the display unit 32 and outputs it.

[0024] <First embodiment: Description of configuration: Display unit> The display unit 32 displays the video signal output from the video processing unit 2 as a video on a screen. The display unit 32 may be, for example, a liquid crystal display (LCD), a thin film transistor (TFT), or an organic light emitting diode (OLED), and any device capable of displaying images can be used as appropriate.

[0025] <First embodiment: Description of configuration: eyepiece lens> The eyepiece 31 projects an image so that the image displayed on the display unit 32 can be viewed at the same magnification. The focal length obtained by adding the focal length of the eyepiece 31 and the focal length of the eyeball determines the viewing angle at which the image displayed on the display unit 32 can be seen. The focal length of the eyepiece 31 is determined so that the angle of view of the image output from the image processor 2 matches the angle of view obtained through the eyepiece 31. At this time, the image is viewed at life-size. The same magnification will be explained later.

[0026] <First embodiment: Description of configuration: Eyecup> The eyecup 33 fixes the distance between the user's eyes and the eyepiece 33 at a predetermined distance. It also blocks light from entering from the sides, making it easier to view the image seen through the eyepiece 33. FIG. 3 is a side view of the AR glasses G of the first embodiment. The AR glasses G have an image display device V installed in the position of one lens of the eyeglass frame GF, and an objective lens 11 in front of the image display device V. The user's face comes into contact with the eyecup 33, and the position of the user's pupil is fixed at a predetermined distance from the objective lens 31. Furthermore, because the contact surface of the eyecup is in close contact with the surface of the face, no light is allowed to enter the eyecup 33 from any angle, be it from the side, above, or below, making it possible to see the image clearly.

[0027] When the distance between the objective lens 31 and the pupil changes, the angle of view of the image viewed through the objective lens 31 changes, and the size of the image perceived by the user also changes; however, by fixing the distance between the objective lens and the pupil with the eyecup 33, the angle of view is fixed at a constant value. The eyecup 33 may be provided with an adjustment mechanism to change the distance between the objective lens 31 and the pupil, thereby adjusting for variations in factors such as the facial contours, pupil position, lens thickness, and eyeball size that differ from person to person.

[0028] <First embodiment: Description of configuration: Communication unit> The communication unit 4 has a wireless communication function, and acquires information from an external communication device to be superimposed on the image processed by the image processing unit 2 , and transmits the acquired information to the image processing unit 2 . It should be noted that the information superimposed on the video is not limited to information acquired from the communication unit 4. Information generated within the video display device V may also be superimposed and displayed.

[0029] The wireless communication function may use a communication standard such as Wi-fi (registered trademark) or Bluetooth (registered trademark), or may be configured using a dedicated communication standard. In addition, communication with other information terminals may be performed via a repeater such as a smartphone, or a connection such as a mesh network may be used. It is desirable that the communication unit 4 uses a communication function that consumes low power.

[0030] Furthermore, the communication unit 4 may transmit the image or information from the image processing unit 2 to an external communication device via wireless communication. The image is an image of the user's field of vision, and the information may be various, such as the user's physical information (pulse, blood pressure, body temperature, respiratory rate, etc.), position information (latitude, longitude, facing direction, elevation angle), and other information for conveying the user's intentions.

[0031] The communication unit 4 may acquire video from an external communication device and transmit it to the video processing unit 2. When the user selects video from an external device, a communication connection is established with the specified external device via the communication unit 4, and video data from the specified external device is acquired. The acquired video data is converted into a video signal for display by the video processing unit 2, and is displayed on the display unit 32.

[0032] <First embodiment: Method for realizing a life-size field of view> FIG. 5 is a diagram illustrating a method for realizing the life-size field of view, which is a feature of the image display device V of the first embodiment. Here, the term "life-size field of view" means that the size of the image seen through the image display device V matches the size of the image seen directly with the eyes.

[0033] The angle of view of the image captured by the image sensor display 12 through the photographing optical system 11 is called the photographing angle of view θv, and is determined by the focal length of the photographing optical system 11 and the size of the image sensor 12. Alternatively, the angle of view of a video signal obtained by trimming a part of the video obtained from the imaging element 12 by the video processing unit 2 may be set as the shooting angle of view θv.

[0034] The viewing angle θf is the angle of the field of view at which the image displayed on the display unit 32 is viewed through the eyepiece lens 31. In the image display device V of the present invention, a life-size field of view is realized by matching the photographing field angle θv with the viewing angle θf. When the image on the image display device V is life-size, even if one eye views an object through the image on the image display device V of the present invention while the other eye views the object directly, the field of view is perceived as natural.

[0035] <First embodiment: explanation of real field of view, video field of view, and synthetic field of view> FIG. 4 is a diagram for explaining the field of view obtained with the AR glasses G of the first embodiment. The field of view FV is the field of view of a car driver when the car is waiting at a traffic light at an intersection. The traffic light is red. A pedestrian is waiting on the sidewalk to the left. A pedestrian is crossing the crosswalk on the opposite side. An oncoming car is waiting for the light.

[0036] The image field of view LV is an image of the field of view visually recognized from the image display device of the first embodiment worn on the left eye. An arrow is superimposed on the image at the location on the road, indicating that the route guidance will require a left turn at the intersection. An icon indicating stop is displayed in the upper left corner of the video field of view LV. A frame to warn people on the sidewalk on the left and on the opposite crosswalk is superimposed on the image of the people, indicating that they are people. A frame is superimposed on the image to warn of stopped oncoming vehicles, indicating that they are cars. The red light is a trigger for the driver to recognize that they are currently in a stopping state and to take the next step, and a frame is superimposed on the red light in the video.

[0037] The real visual field RV is an area including the right half of the visual field FV and indicates the visual field of the right eye. The left side of the real field of view (RV) is visible to the right eye. As such, there is an area near the center of the field of view where the right eye's field of view and the left eye's field of view overlap.

[0038] The synthetic visual field SV indicates the visual field recognized by the user of the AR glasses G of the first embodiment. The image seen from the left eye and the real view seen from the right eye appear to be the same size, so they are synthesized in the brain and perceived as a natural view. The area near the center of the synthetic visual field SV is visible in both the real visual field RV and the image visual field LV, so the user feels that it is semi-transparent. In this way, in the area where the visual fields of both eyes overlap, the visual fields of both eyes are complementary to each other and are visually recognized. For example, although there is a delay in the visual field of view (LV), movement can be recognized in the real field of view (RV), and even if the image in the visual field of view (LV) is distorted due to intense movement, the real field of view (RV) can compensate for this.

[0039] As described above, users are provided with a wide, natural field of view, and feel as if information is displayed in that field. By placing the superimposed information display at the edge of the field of view or in an area determined to contain nothing noteworthy from the image, the view is not obstructed and the user does not feel uncomfortable. Furthermore, by making it easier to notice an object that requires attention by adding a frame to it, it is possible to prevent the user from overlooking it.

[0040] <Effects of the First Embodiment> As described above, the AR glasses G of the first embodiment can provide a natural visual field for one eye, even when the visual field is an image. This means that if one eye sees a real visual field RV seen directly and the other eye sees an image visual field LV through an image displayed on the image display device V, the user will not feel any discomfort as long as the image sizes are the same. Furthermore, by displaying the information on the image display device V, the information is visually recognized as information superimposed on the actual field of view. Moreover, by switching the image displayed on the image display device V to an image from an external device, it is possible to easily check an image from a different viewpoint.

[0041] <Second embodiment: Overview> The image display device V of the second embodiment includes a second imaging system in addition to a first imaging system. Here, the imaging system is a general term for the imaging optical system, the imaging element, and the image processing unit. The image display device V of the second embodiment can switch to and display an image with different characteristics acquired from a second imaging system, in addition to an image with a field of view of life size.

[0042] <Second embodiment: configuration> FIG. 6 is a block diagram showing the functional configuration of the video display device V of the second embodiment. The image display device V of the second embodiment includes a second imaging optical system 31, a second image pickup element 14, a second image processing unit, and a display switching unit 5 in addition to the components of the first embodiment. The first photographing optical system 11, the first image sensor 12, and the first image processing unit 21 are the same as those in the first embodiment. Hereinafter, the description of the same configuration as in the first embodiment will be omitted, and only the different configuration will be described.

[0043] Second Embodiment: Second Imaging Optical System The second photographing optical system is an optical system with different characteristics from the first photographing optical system. For example, the second imaging optical system has a longer focal length than the first imaging optical system, and can capture telephoto images. Furthermore, the second imaging optical system has a focal length shorter than that of the first imaging optical system, and can capture an image with a wider angle than the field of view. The second imaging optical system is filtered to transmit only light in a specific band. The second photographing optical system may have the same optical characteristics as the first photographing optical system but may be oriented in a different direction. The characteristics of the second imaging optical system are not limited to those described above, and various other optical systems are also possible.

[0044] <Second embodiment: Configuration of second image sensor> The second imaging element may be the same as the first imaging element, or may be an imaging element of a different size. The second imaging element may have a characteristic of being highly sensitive to light in a specific band.

[0045] <Second embodiment: Configuration of second video processing unit> The second image processor 22 reads out the electrical signal converted by the second imaging element as an image signal, performs image processing according to the characteristics of the second imaging optical system and the characteristics of the second imaging element, and outputs the result as an image signal. For example, when the second optical system and the second imaging element have the characteristics of capturing an infrared image, areas with high brightness may be colored in warm colors and areas with low brightness in cool colors according to the brightness.

[0046] <Second embodiment: Configuration of display switching unit> The display switching unit 5 receives the video signal from the first video processing unit 21 and the video signal from the second video processing unit 22, and switches between the video signals to be displayed on the display unit 32 based on a user's instruction, and outputs the switched video signals. The switching may be performed by providing a switch on the surface of the housing of the image display device V, or may be performed when a predetermined gesture is detected at a predetermined position in the image signal from the first image processing unit 21. There are various other methods of instructing video switching, and any method may be used as long as it can convey the user's intention to the video display device V.

[0047] <Modification of the second embodiment> The first video processing section and the second video processing section may be integrated into one unit. In this case, the integrated video processing unit 2 only needs to perform the video processing selected by the user, and the display switching unit 5 is not necessary.

[0048] <Second embodiment: exterior> FIG. 7 is a diagram showing the outer shape of the AR glasses G according to the second embodiment. The AR glasses G have a lens GL on the right eye side of the eyeglass frame GF, allowing the viewer to see through it. An image display device V is installed on the left eye side, and the position of the pupil is fixed by an eyecup 33. A first imaging optical system 11 and a second imaging optical system 13 are installed in front of the image display device V, and are configured to capture images in the direction of the user's field of view. The optical axis of the second imaging optical system does not need to be oriented in the same direction as the optical axis of the first imaging optical system, and may be oriented laterally or rearward.

[0049] <Effects of the Second Embodiment> As described above, the image display device V of the second embodiment can switch from displaying the field of view at the same magnification to an image with other characteristics, so that it is possible to switch to an image with different characteristics without changing the device. For example, you can switch from a normal field of view to telephoto footage, wide-angle footage, thermal camera footage, infrared camera footage, or rear view footage, allowing you to view images with different characteristics without changing equipment. [Explanation of symbols]

[0050] G AR glasses, GF eyeglass frames, V video display device 11 photographing optical system, 12 image sensor, 2 image processing unit, 31 eyepiece, 32 display unit, 33 Eyecup, 4 Communication Unit,

Claims

1. a first photographing optical system that forms a subject image; a first image sensor that converts a subject image formed by the first photographing optical system into an electrical signal; a first image processing unit that converts the electrical signal from the first imaging element into an image signal; a display unit that displays the video signal converted by the video processing unit; an eyepiece for observing the image displayed on the display unit at a magnification equal to the field of view;

2. The video display device according to claim 1 further comprises a communication unit that acquires information from an external device via wireless communication, The first video processing unit superimposes the information acquired by the communication unit on the converted video signal.

3. The image display device according to claim 1 includes: a second photographing optical system that forms an image of a subject; a second image sensor that converts the subject image formed by the second photographing optical system into an electrical signal; a second image processing unit that converts the electrical signal from the second imaging element into an image signal; The display device further comprises a display switching unit that switches between the image from the first image processing unit and the image from the second image processing unit based on an instruction from a user and outputs the image to the display unit.

4. In the video display device according to any one of claims 1 to 3, the communication unit acquires video information from an external device, The first video processing unit converts the video information acquired from the communication unit into a video signal that can be displayed on the display unit, and displays the video information on the display unit.

5. AR glasses comprising a frame for fixing the image display device according to any one of claims 1 to 4 to one of the eyes.

Images