Augmented optical path for land based vehicle periscope

Abstract

A vehicle periscope assembly provides for real time augmentation of viewing objects viewable within an optical viewing path. A display is mounted proximate a defined optical viewing path and is configured to generate images that correspond to objects viewable along the optical viewing path.

Description

AUGMENTED OPTICAL PATH FOR LAND BASED VEHICLE PERISCOPECROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Application Serial No. 63 / 568,644 filed on March 22, 2024.TECHNICAL FIELD

[0002] The present disclosure relates generally to a periscope for a land based military vehicle, and more particularly to a periscope with an augmented optical path.BACKGROUND

[0003] Armored land base vehicles may include a periscope to provide safe viewing for vehicle operators and occupants. A typical periscope includes two reflective surfaces spaced apart to define an optical viewing path. The reflective surfaces are angled appropriately to present a view of the external environment to a vehicle occupant within the vehicle. A vehicle operator guiding a vehicle may need vehicle operation information that may be provided by gauges or displays within the vehicle. Viewing of those gauges may require the vehicle operator to look away from the periscope view of the outside environment. Shifting of a view away from the outside can complicate vehicle maneuvering and is therefore desirable to minimize.SUMMARY

[0004] A vehicle periscope assembly according to one example disclosed embodiment includes, among other possible things, a housing defining mounting features, a top opening and a viewing opening. A top reflective surface is supported relative to a bottom reflective surface. The top reflective surface and the bottom reflective surface are aligned to define the optical viewing path between the top opening and the viewing opening. A display is provided and configured to generate an image that corresponds to an object viewable along the optical viewing path and project the generated image into the optical viewing path. A controller is programmed to operate the display and generate a digital image to augment in real time the optical viewing path.

[0005] Although the different examples have the specific components shown in the illustrations, embodiments of this disclosure are not limited to those particular combinations. It is possible to use some of the components or features from one of the examples in combination with features or components from another one of the examples.

[0006] These and other features disclosed herein can be best understood from the following specification and drawings, the following of which is a brief description.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a front view of an example periscope assembly embodiment.

[0008] Figure 2A is a schematic view of the example periscope assembly embodiment including an internal cavity.

[0009] Figure 2B is a schematic view of the example periscope assembly including an optical prism.

[0010] Figure 3 is a schematic view of another example periscope assembly embodiment.

[0011] Figure 4 is a schematic view of the example periscope assembly embodiment relative to an object.

[0012] Figure 5 is a schematic view of an example augmented view of an object.

[0013] Figure 6 is a schematic view of an example controller embodiment.

[0014] Figure 7 is a flow diagram of an example method of operation of the example periscope assembly.

[0015] Figure 8 is a schematic view of another example periscope embodiment.

[0016] Figure 9 is a schematic view of an alternative modification of the embodiment shown in Figure 8.

[0017] Figure 10 is a schematic view of yet another periscope embodiment.DETAILED DESCRIPTION

[0018] Figures 1 and 2A schematically illustrate a periscope assembly 20 for use in viewing an outside environment from an enclosed position, such as when operating a vehicle. The example periscope assembly 20 provides for the generation of an augmented view of objects visible along an optical viewing path. The augmented view providesadditional information relating to features of a viewed object and may also provide information indicative of vehicle operation. The example periscope assembly 20 provides the additional benefit of driving the vehicle with zero latency of the external view, thus preventing motion sickness, while retaining the ability to superimpose digital information over the optical path image.

[0019] Figure 2A illustrates the example periscope assembly 20 as including a housing 22 that defines an internal cavity 24, a top opening 26 and a viewing opening 28. The top opening 26 and the viewing opening 28 are disposed on opposite sides of the housing 22.

[0020] Figure 2B illustrates an alternate configuration of the example periscope assembly 20 where an optical prism 25 is utilized between the top reflective surface 30 and the bottom reflective surface 34 instead of an open space through the internal cavity 24. The internal cavity 24 (Figure 2A) or the optical prism 25 (Figure 2B) may be utilized interchangeably within the scope and contemplation of this disclosure.

[0021] Referring to Figures 1, 2A and 2B, the top reflective surface 30 is disposed proximate the top opening 26 and is aligned with a bottom reflective surface 34. The top reflective surface 30 and the bottom reflective surface 34 are angled relative to each other to define an optical viewing path 40 from the top opening 26 to the viewing opening 28. The example top reflective surface 30 is part of top prism 32 that includes a back side 68. The example bottom reflective surface 34 is part of a bottom prism 36.

[0022] A display 38 is mounted proximate the top reflective surface 30 and is configured to generate an image that is viewable along the optical viewing path 40. In one disclosed example, the display 38 is mounted to a back side 68 of the example top prism 32.

[0023] Referring to Figure 3, the display 38 may also be mounted proximate to the bottom reflective surface 34. In one example embodiment, the bottom reflective surface 34 is part of a prism 36 that includes a back side 70. The display 38 is mounted to the back side 70 of the prism 36. Although examples of mounting locations of the display 38 are shown by way of example, other locations that enable images generated by the display to be projected into the optical viewing path 40 may also be utilized and are within the contemplation and scope of this disclosure. As the display is brighter than ambient lightsources, the light from the display is perceived by the user more predominantly than the ambient light source.

[0024] Referring to Figure 4 with continued reference to Figures 2 and 3, operation of the display 38 is governed by a controller 44 associated with the display 38. The controller 44 is shown schematically and may be mounted as part of the periscope assembly 20 and / or may be provided separate from the periscope assembly 20. The example controller 44 receives information as indicated at 58 from related external inputs, such as for example from a vehicle controller (not shown). Information 58 received by the controller 44 may include vehicle operating information, external communications, and / or any other information relevant to vehicle operation.

[0025] The camera 46 captures images of a surrounding area and communicates those images to the controller 44. The controller 44 uses images of objects captured by the camera 46 to control the display 38 and generate images schematically shown by arrow 42. The images 42 are visible through the reflective surface 30 and are projected within the optical viewing path 40 that is visible by an operator. In one example embodiment, the display 38 generates images 42 to correspond with an object viewable within the optical viewing path. The controller 44 is programed to operate the display 38 to generate the digital image 42 such that it corresponds to the object 48.

[0026] Referring to Figures 4 and 5, the example periscope assembly 20 and an example view through the viewing opening 28 are shown schematically in view of an object 48. The example object 48 is any feature, vehicle, part, landscape feature, or object that is viewable along the optical viewing path 40. The camera 46 is disposed at a known location relative to the periscope assembly 20. The controller 44 is programmed to analyze images obtained by the camera 46 and map a location of the object 48 in the captured images relative to a location that the object is shown within the optical viewing path 40. In this example, a relative location 72 of the camera 46 relative to the periscope assembly 20 is known and used to map images into the optical viewing path 40.

[0027] The example system includes a software layer that utilizes machine vision algorithms. In one example embodiment, an object detection algorithm is included and stored in a memory device of the controller 44. In one example, the machine vision system utilizes a camera of sufficient resolution to provide enough data to identify objects, and then displaysthe captured video image with bounding boxes or other indicators to identify the detected objects. The example system further provides for a module, algorithm and / or software instructions that delete the video feed from what is displayed on the digital display, leaving only the bounding boxes (or other identifiers) on the display. The system or another system is then utilized (either physical or digital) to calibrate the display to the optical path images such that the objects within the optical path are located similar to where they would be in the digital display prior to video deletion, thusly ensuring that the bounding boxes are located appropriately on the optical path image.

[0028] The optical viewing path 40 provides an augmented view, in real time, of objects 48. In one disclosed example, the object 48 is shown with a generated image 50. The example image is a bounding box 50 and outlines the image. As appreciated, the shape, size, color, and graphical image may be different than the simple example illustrated in Figure 5. The type, size, shape, and color of the image 50 may be modified and adapted based on predefined preferences, criteria, and application.

[0029] The images generated by the display 38 may also include values as indicated at 54 relating to the object 48. The values 54 may represent any relevant value that would be desired and useful to a vehicle operator. For example, the values, may be distance to the object, size of the object, speed or any other variable parameter that would be useful to the vehicle operator.

[0030] The images generated by the display 38 may also include information 52 relating to operation of the vehicle or some other information that may not be specifically associated with the object 48. For example, the information 52 may include direction, speed, time, status, warnings, or any other operational parameter that would be useful to a vehicle operator. The display of such information 52 enables a vehicle operator to maintain outward viewing while still receiving information needed for vehicle operation. Moreover, other machine vision information such as distance to an object and any other information that could be useful to an occupant of a vehicle.

[0031] Further embodiments of the disclosed system and assembly may include utilization of lenses or an additional set of mirrors to increase the perceived or physical distance between the digital display and the optical path images, such that the bounding boxes or other data overlays appear within the same or similar focal plane to the optical pathimages to reduce the need for the eyes to refocus between the augment reality overlay and the optical path image.

[0032] Referring to Figure 6, the example controller 44 includes a processor 60, a memory device 62 and an input / output device 64. The memory may store one or more programs executable by the processor 60 to analyze images received by the camera 46 and generate an output 74 to control generation of images 42 by the display 38.

[0033] The controller 44 receives information from the camera 46 and from other information sources as schematically shown at 58. Additionally, the controller 44 may receive or store database information 66 that is used for analysis of the images and determination of what images to generate by the display 38.

[0034] The example controller 44 is programmed to perform machine vision functions to recognize objects within the captured images and command the display 38 to generate images based on recognized characteristics of the object.

[0035] Referring to Figure 7 with continued reference to Figures 4, 5 and 6, an example operational embodiment of the example system is schematically shown. In an initial step, a calibration between a camera and an optical path including mirrors / lenses of the periscope is performed such that images align when viewed by a user both on the digital display and the optical path mirrors.

[0036] The machine vision algorithm will perform a function upon a video stream produced by one or more cameras. In one example embodiment, an object detection algorithm is utilized whereby an object (e.g. a human) is detected by the machine vision algorithm. The object is bounded by an identifying feature, such as a bounding box of a bright color. This bounding box is then displayed overlaid on the video feed. Additional information may be included with the bounding box like range data if used with either stereo vision cameras that detect range through geometry, or other range finding devices.

[0037] The overlaid video feed is then fed to the digital display. In one example, the digital video is deleted, but the bounding boxes or other identifying features and information is retained and displayed on the digital display. The digital display will then be projected into the optical path image as transferred through the periscope with a series of mirrors.

[0038] The digital display image is projected by the use of semitransparent mirrors also commonly known as beam splitters to merge the two image paths to the user’s eyes. At some point prior to use, during manufacturing or installation, a calibration function will have been performed to ensure that the images detected by the camera(s) are properly aligned with the images as seen through the periscope optical path. The calibration function may be done mechanically / physically, by putting an object at a known distance and then projecting both the full video feed and the optical path image into the users view path. The camera is then adjusted physically until the two images align. Once calibrated as desired, the camera will be securely mounted in place.

[0039] A digital calibration may also be utilized. For a digital calibration, the image is cropped in some way to ensure that the images align. The periscope may have a different aspect ratio as compared to the digital camera sensors. Accordingly, for digital alignment, it desired for the camera sensors have sufficient resolution that cropping and scaling (digital zoom function) results in enough pixel density to properly conduct machine vision on the resultant image. Once the two images are aligned, the data outside of the new field of view, matching the optical path field of view, is discarded.

[0040] In one example operational embodiment, the controller 44 receives a captured image of a surrounding environment from at least one camera 46. Although a single camera 46 is shown by way of example, multiple cameras 46 may be utilized and provide information to the controller 44.

[0041] The controller 44 uses information in the image and information 58 from other sources to map the captured image from the camera 46 to a corresponding location in an optical viewing path 40. Because the camera 46 and the periscope assembly 20 may be located at different points on a vehicle, any object viewed through the periscope assembly 20 may differ in appearance from a captured image. Moreover, the perspective and distance to the object, periscope assembly 20 and camera 46 will differ. The example controller 44 is programmed to map the location of the object with the image to a location on the display 38 that will provide a corresponding location within the optical viewing path 40. Mapping by the controller 44 may done based on a known, predefined relative position of the camera 46 relative to the periscope assembly 20. Mapping may also be done based on features in theimage that are indicative of an identifiable perspective that can be matched to the image along the optical viewing path 40.

[0042] The controller 44 facilitates the generation of a digital image on the display 38 that corresponds with the captured image. Once the location of the object is mapped to the optical viewing path, the controller 44 determined what type of image is generated. The type of image may be of any size, shape, color, or location in the optical viewing path. The configuration of the generated image may be predefined and stored in the database 66. The database 66 may store information pertaining to the geometry, size, shape, color, and any other format for the generated image. The generated image may be overlaid over the object, disposed near the object, or otherwise arranged as desired and in consideration to the information conveyed.

[0043] Once the image is configured based on the object characteristics, the image is projected into the optical viewing path 40 in a location corresponding with a location of an object viewable within the optical viewing path 40. The example display 38 is disposed behind either of the reflective surfaces 30, 32. The display 38 is operated at a predefined contrast ratio and screen brightness that is determined to provide for viewing of the generated image while not obscuring the optical viewing path 40. In other words, the generated image 42 is visible to an operator, but does not block out or otherwise overtake viewing of objects along the optical viewing path 40.

[0044] Referring to Figure 8, another example periscope system is shown and generally indicated at 76. The periscope system 76 includes a normal reflective mirror 84, a semi-transparent mirror 82, a lens 80, and a digital display 78. The addition of the lens 80 provides for images displayed upon the digital display 78 to be altered by passing the light path through the lens 80 to create an image in a similar focal plane as the optical path images as reflected by the semi-transparent mirror 82, as perceived by the user 86.

[0045] Referring to Figure 9 with continued reference to Figure 8, in one example embodiment, the lens 80 is movable relative to the display 78 to adjust the clarity of images projected on the semi-transparent mirror 82. The lens 80 may be coupled to an actuator 102 that is controlled through instructions 108 from a controller 44. The actuator 102 is operable to move the lens 80 relative to the display 78 as indicated by arrows 104 to adjust a distance106 therebetween. Adjustment of the distance 106 provides for focusing the image on the reflective mirror 84 to provide a clear image for viewing by a user 86.

[0046] Referring to Figure 10, another example periscope system embodiment 88 includes a normal reflective mirror 96, a semi-transparent mirror 94, a set of normal reflective mirrors 90, 92, and a digital display 98. The disclosed arrangement provides for the focal plane of the images displayed upon the digital display 98 to be altered by increasing the physical distance between the viewers eyes and the digital display 98 to create an image in a similar focal plane as the optical path images as reflected by the semi-transparent mirror 94 and the normal reflective mirror 96, as perceived by the user 100.

[0047] Accordingly, the example periscope assembly 20 provides real time augmentation of objects viewable along an optical viewing path 40.

[0048] Although an example embodiment has been disclosed, a worker of ordinary skill in this art would recognize that certain modifications would come within the scope of this disclosure. For that reason, the following claims should be studied to determine the scope and content of this disclosure.

Claims

CLAIMSWhat is claimed is:

1. A vehicle periscope assembly comprising: a housing defining mounting features, a top opening and a viewing opening; a top reflective surface supported within the housing; a bottom reflective surface supported within the housing, wherein the top reflective surface and the bottom reflective surface are aligned to define an optical viewing path between the top opening and the viewing opening; a display configured to generate an image that corresponds to an object viewable along the optical viewing path and project the generated image into the optical viewing path; and a controller programmed to operate the display to generate a digital image corresponding to the object based on an external input of information.

2. The vehicle periscope assembly as recited in claim 1, wherein the display is further configured to generate a secondary image providing information indicative of a vehicle operating condition.

3. The vehicle periscope assembly as recited in claim 1, further including a camera configured to obtain captured images of an external environment including those objects within the optical viewing path.

4. The vehicle periscope assembly as recited in claim 3, wherein the controller is further programmed to receive and analyze captured images obtained from a camera and to generate images on the display that correspond to the objects in the optical view path.

5. The vehicle periscope assembly as recited in claim 3, wherein the controller is further programmed to analyze images obtained by the camera and map a location of the object in the captured images relative to a location of the object within the optical viewing path.

6. The vehicle periscope assembly as recited in claim 5, wherein the controller is further programmed to determine a characteristic of the object in the captured images and to operate the display to generate an image corresponding to the characteristic.

7. The vehicle periscope assembly as recited in claim 6, wherein the characteristic is variable and the generated image varies to correspond with variations in the characteristic.

8. The vehicle periscope assembly as recited in claim 1, wherein the display is mounted proximate to the top reflective surface and the top reflective surface is semi-transparent such that upon actuation of the display the generated image is viewable through the top reflective surface as part of the optical viewing path.

9. The vehicle periscope assembly as recited in claim 1, wherein the display is mounted proximate to the bottom reflective surface and the bottom reflective surface is semitransparent such that upon actuation of the display the generated image is viewable through the bottom reflective surface as part of the optical viewing path.

10. The vehicle periscope assembly as recited in claim 1, wherein the display is configured to generate an image with a predefined contrast ratio, wherein the predefined contrast ratio and screen brightness provides for viewing of the generated image through one of the top reflective surface and the bottom reflective surface.

11. The vehicle periscope assembly as recited in claim 1, wherein the controller is configured to control generation of the image on the display to match the generated image with a focal plane of the object within the optical viewing path.

12. The vehicle periscope assembly as recited in claim 1, wherein the controller is programmed to control operation of the display to provide an optical viewing mode where the display is turned off and an augmented viewing mode where the display is on and generating at least one image corresponding to an object within the viewing path.

13. The vehicle periscope assembly as recited in claim 1, further comprising a lens for altering a focal point of images generated by the display to create a similar focal plane as the optical path images.

14. The vehicle periscope assembly as recited in claim 13, further comprising an actuator configured to move the lens relative to the display to adjust the focal plane.

15. The vehicle periscope assembly as recited in claim 1, wherein the digital display is movable to adjust a physical distance of the digital display such that images generated by the display may be aligned with images in the optical path.

16. A vehicle periscope assembly comprising: a housing defining mounting features, a top opening, and a viewing opening; a top reflective surface supported within the housing; a bottom reflective surface supported within the housing, wherein the top reflective surface and the bottom reflective surface are aligned to define an optical viewing path between the top opening and the viewing opening; a display configured to generate an image that corresponds to an object viewable along the optical viewing path and project the generated image into the optical viewing path, wherein at least one of the top reflective surface and the bottom reflective surface passively provides for the generated image on the display to be visible within the optical viewing path; and a controller programmed to receive and analyze a captured image obtained from a camera and to generate images on the display that correspond to the objects in the optical view path.

17. The vehicle periscope as recited in claim 16, wherein the top reflective surface is part of a top prism that includes the top reflective surface and a back side, and wherein the display is mounted to the back side of the top prism.

18. The vehicle periscope assembly as recited in claim 16, wherein bottom reflective surface is part of a bottom prism that includes the bottom reflective surface and a back side, and the display is mounted to the back side of the bottom prism.

19. The vehicle periscope assembly as recited in claim 16, wherein the controller is further programmed to map a location of the object in the captured image relative to a location of the object within the optical viewing path.

20. A method of generating an augmented image through a vehicle periscope assembly, the method comprising: obtaining a captured image of a surrounding environment from at least one camera; mapping the captured image from the camera to a corresponding location in an optical viewing path; generating a digital image on a display device that corresponds with the captured image; and projecting the generated image into the optical viewing path in a location corresponding with a location of an object viewable within the optical viewing path.

21. The method as recited in claim 20, further comprising analyzing the captured image to determine a characteristic of an object within the captured image that corresponds with the object viewable within the optical viewing path and generating the image based on the determination.

22. The method as recited in claim 20, wherein the determined characteristic comprises at least one of a speed, distance, and object category.

23. The method as recited in claim 20, further comprising generating an image relating to a vehicle operating parameter and projecting the image indicative of the vehicle operating parameter into the optical viewing path.

24. A method of calibrating a digital display image to align with an image within an optical path, the method comprising: generating an image on a display device of an object at a known distance; adjusting an image capture device to align the generated image from the display device to an image viewable along an optical viewing path; and securing a camera in a location corresponding with alignment between the generated image and the optical viewing path.

25. A method of calibrating a digital display image to align with an image within an optical path, the method comprising: generating an image on a display device of an object at a known distance; matching the object in the generated image with the object as viewed through an optical viewing path; aligning the generated image with the optical image based on the object; removing portions of the generated image not viewable along the optical viewing path; and formatting future generated images based on the previous alignment and removed portions.

Images