Smart glass for vehicles

Abstract

A system for displaying information to a user of a vehicle is provided, the system comprising smart glass for displaying information as three-dimensional (3D) elements, a navigation assistance component, and a central database (104), wherein the smart glasses (102) and the navigation assistance component communicate with the central database (104). The database (104) stores information and facilitates sending and displaying relevant and updated information on the smart glass (102) in real time depending on the location through which the vehicle is being maneuvered. Furthermore, the information is displayed on the smart glass (102) through markerless augmented reality (AR) technology, thereby eliminating the need for an external device or projector for displaying the information.

Description

Technical Field

[0001] The present invention relates to a smart glass for a vehicle, and more particularly to a smart glass that functions based on markerless augmented reality. Background Art

[0002] The Background Description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.

[0003] The windshield or windshield of an automobile is the front window that provides visibility to the driver while driving. Conventional windshields protect the vehicle's occupants from wind and flying debris, dust, insects, rocks, and provide an aerodynamically shaped window toward the front of the vehicle.

[0004] Road signs (including directional signs, warning signs, billboards, traffic lights, road markings and protective barriers) occupy a large space on the road, and the costs incurred in manufacturing, importing, installing and maintaining these signs or signs are expensive. In addition, damaged road signs may be an eyesore, and in some cases, it may threaten safety on the road, for example, vehicles tend to hit these signs during road accidents. It should be noted that these signs are installed after the road is paved and are usually used for drivers rather than pedestrians.

[0005] Augmented reality is an interactive experience of a real-world environment, where objects residing in the real world are enhanced by computer-generated sensory information, sometimes across multiple sensory modalities, including vision, hearing, touch, body sensation, and smell, augmented reality is used to enhance the natural environment or situation and provide a perceptually rich experience. With the help of advanced AR technology (e.g., adding computer vision, incorporating AR cameras into smartphone applications and object recognition), information about the user's surrounding real world becomes interactive and digitally manipulated. Information about the environment and its objects is overlaid on the real world. The information can be virtual or real.

[0006] Head-up displays (HUDs) based on augmented reality are currently being used on windshields. A head-up display (HUD) is a multimedia system that projects data on the windshield using a projector device or any additional device that can help project information on the windshield. However, this is very disadvantageous because these devices for projecting information are distracting and severely obstruct normal vision, thereby posing a threat to the driver. Other disadvantages associated with HUDs include limited intelligence, short lifespan, and low light efficiency, which creates difficulties for the driver at night.

[0007] Therefore, there is a need for a system that eliminates the need for an external projector device while displaying useful information to the driver via the windshield. There is also a need for a system for displaying desired information to the driver without actually having to manufacture, install, or maintain physical signage, by which a significant amount of money can be saved. Summary of the invention

[0008] It is therefore an object of the present invention to propose a system which eliminates the need for an external projector device while displaying useful information to the driver via the windshield and thereby eliminates the need to manufacture, install and maintain physical signage on the road.

[0009] The present invention relates to a system for displaying information to a user of a vehicle, the system comprising smart glass for displaying the information as three-dimensional (3D) elements, a navigation assistance component and a central database, wherein the smart glass and the navigation assistance component communicate with the central database, which stores the information and helps to send and display relevant and updated information on the smart glass in real time depending on the location through which the vehicle is being maneuvered.

[0010] In another embodiment of the present invention, the displayed information includes road signs, obstacles or advertisements that are relevant to the location through which the vehicle is being maneuvered.

[0011] In another embodiment of the present invention, artificial intelligence (AI) and augmented reality (AR) technologies are used to display information on smart glass.

[0012] In another embodiment of the present invention, marker-less augmented reality (AR) technology is used to display information on smart glass, thereby eliminating the need for an external device or projector for displaying information.

[0013] In another embodiment of the present invention, the navigation assistance component is a Global Positioning System (GPS) component.

[0014] In another embodiment of the present invention, the system further includes multiple sensor components and multiple cameras located at multiple locations on the vehicle and on the smart glass to achieve complete coverage of the surrounding environment.

[0015] In another embodiment of the present invention, the plurality of sensor components include position sensors, motion sensors, light and acceleration sensors.

[0016] In another embodiment of the present invention, relevant and updated signals are displayed in real-time on smart glass, eliminating the need to install physical signage on the road.

[0017] In another embodiment of the present invention, the plurality of cameras have infrared (IR) imaging capabilities.

[0018] In another embodiment of the invention, the central database is managed by an entity which regularly updates the central database with new road signs or warnings.

[0019] In another embodiment of the present invention, the system further comprises a central processing unit (CPU) programmed to process data received from the navigation assistance component and the plurality of cameras; and to generate and display relevant information on the smart glass connected to the central database.

[0020] In another embodiment of the present invention, a light and acceleration sensor measures illuminance values ​​from the surrounding environment and then matches the measured illuminance values ​​when illuminating elements displayed on the smart glass.

[0021] In another embodiment of the present invention, the system further comprises a 3D digital compass.

[0022] In another embodiment of the present invention, the smart glass is automatically activated when a vehicle including the smart glass is started. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The subject matter that is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other aspects, features and advantages of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0024] Figure 1 is a block diagram of a smart glass system according to the present invention.

[0025] FIG. 2( a ) shows a normal windshield without the smart glass according to the present invention (when driving on a new road without road facilities).

[0026] FIG. 2( b ) shows how objects are displayed on the smart glass according to the present invention (when driving on a new road without road facilities). DETAILED DESCRIPTION

[0027] Will combine Figure 1 -2 describes aspects of smart glass that functions based on markerless augmented reality according to the present invention. In the detailed description, reference is made to the accompanying drawings, which form a part of the present invention and show by way of illustration specific embodiments in which the invention may be practiced. It should be understood that other embodiments may be utilized and logical changes may be made without departing from the scope of the present invention. Therefore, the following detailed description should not be construed as limiting, and the scope of the present invention is defined by the appended claims.

[0028] The proposed smart glass according to the present invention is positioned to cover or be fixed to replace the entire windshield of the vehicle on which the smart glass is implemented. Therefore, the user or driver of the vehicle with smart glass observes the outdoor environment or road through the smart glass on the vehicle windshield. The implementation of the proposed smart glass on the vehicle results in an intelligent and visually attractive road or highway, eliminating the need for guide signs, warning signs or advertising signs on the road. However, the driver or user inside the vehicle where the smart glass is implemented will continue to observe the required warning or direction signs, required obstacles, etc. through the smart glass, thereby ensuring road safety while creating a new environment and enhancing the aesthetics of the road and its associated surrounding areas.

[0029] The proposed strategic view of smart glass in vehicles is consistent with the future strategic view of global industrial products, which is oriented towards using artificial intelligence (AI) and augmented reality (AR) to achieve greater productivity. This productivity is achieved without significant resource consumption and leads to significant savings in road engineering and road maintenance by eliminating street furniture (defined as objects or equipment installed along streets and roads for various purposes) and the associated manufacturing, installation, transportation and maintenance work.

[0030] In a preferred embodiment of the present invention, the smart glass is designed to display elements in three dimensions (3D) through markerless augmented reality (AR) technology, thereby eliminating the need for any external device or projector for performing this function. The displayed elements include all required road signs - traffic signals, guidance or direction signs, warning signs and restriction signs. In another embodiment, the smart glass also displays advertisements (traditionally projected on the roadside using billboards or billboards) in a way that the driver's attention or vision remains unobstructed. Smart glass is not applied to ordinary and current automotive glass, but a complete replacement of ordinary glass that does not meet the working specifications of augmented reality technology, because the smart glass designed for the present invention is a 3D vision covering the user's central field of view and serves as a transit station to allow the user's eyes to view the real world as it is, but in an enhanced manner with the help of augmented reality technology to enable the driver or user to see elements added to the real road in the form of triple object size.

[0031] If used Figure 1As depicted in the block diagram in FIG. 1 , the system for implementing the smart glass 102 in the vehicle also includes components such as a global positioning system (GPS) navigation component 112, a plurality of sensor components 110 including position detection and mapping sensors, motion and acceleration sensors, and a camera 108 with infrared (IR) imaging capabilities and a 3D digital compass. All additional components (external or internal) that help enhance the functionality of the smart glass are manufactured and positioned in a manner that the driver's field of view is not impaired in any way. Therefore, all components are designed to be incorporated in or near the smart glass, so that the smart glass is designed, manufactured, and installed as a single piece or device in combination with all other required components.

[0032] In another embodiment, multiple cameras 108 work in combination with smart glass. These cameras are located at various locations on the vehicle and on the smart glass 102 to achieve 360° coverage of the surrounding environment. Therefore, multiple cameras are driven by 3D imaging technology including infrared and RGB specifications with depth sensing capabilities - to enhance monitoring of the environment and provide a safer driving experience. In addition, the screen for smart glasses implementation is designed to support 3D vision, covering the driver's central field of view - the central field of view is temporarily 90° from the central gaze point, 50° above and nasal, and 60° below.

[0033] In another embodiment of the invention, the system further comprises a plurality of sensor components and a plurality of cameras located at a plurality of locations on the smart glass so as to be able to fully cover the surroundings (geometrically and technically distributed within the perimeter of the smart automotive glass) without any impact on the driver's field of view. In another embodiment of the invention, the external parts and components of the device are manufactured and positioned inside or around the glass as a single piece connected to the automotive glass (in the form of a flat plate) so as not to affect the driver's field of view through the glass.

[0034] The smart glass is in constant communication with a central database 104 managed by the traffic department, which regularly updates the database 104 with new road signs or warnings. The database 104 stores all types of traffic signals and road signs and helps send appropriate and updated signals in real time and display appropriate and updated signals on the smart glass depending on where the vehicle is being maneuvered. A portion of this database can also be accessed and managed by commercial parties interested in using the smart glass for advertising. All data displayed on the smart glass is in the form of 3D models or elements, which gives the user the illusion that these elements are actually still present on the physical road.

[0035] In another preferred embodiment, the difference in the user's or driver's perspective will not affect the clarity, or how the elements are perceived or interpreted by the user or driver to be displayed on the smart glass. In other words, a person viewing the external environment through the smart glass (from any position or angle within the vehicle) will be able to view all elements displayed on the smart glass with the same clarity and accuracy. The basic principle behind the working of the smart glass according to the present invention is markerless augmented reality technology. This technology enables 3D content to be overlaid directly onto a screen or any other physical platform, where the augmentation occurs in real time and within the context of the current environment (rather than creating an artificial environment to replace the real environment with a virtual environment).

[0036] According to the present invention, all the required information (traffic signals, road signs and direction signs) that should be perceived and viewed by the driver are displayed on the smart glass 102 when needed. There is a central processing unit (CPU) 106, which is programmed to generate and display 3D virtual objects on the smart glass 102, and has the processing power to support a high-power graphics processor such as Qualcomm (XR1 processor). When starting a vehicle containing the smart glass according to the present invention, the smart glass is also automatically activated and does not need to be turned on or started separately by the driver or user of the smart glass. This is mainly because the smart glass is a centrally operated device with a centrally controlled and managed database-the database cannot be changed or tampered with by external or unauthorized individuals. In addition, the smart glass is programmed to be compatible and can be connected to any operating system.

[0037] A user driving to a destination using a vehicle implemented with the smart glass according to the present invention will not need prior knowledge about the roads or associated environments that the user may encounter on the way to the destination. When starting the vehicle, because the AR technology used in the present invention (specifically, technology without tags in reality) - tagless AR is location-based and supports simultaneous positioning, mapping and special sensors and cameras, the smart glass is simultaneously started and all associated external and internal components that work with the smart glass are activated. In another embodiment of the present invention, due to the usual large size and rolling when manufacturing automotive glass, it is easy to manufacture the device as a whole to connect all parts of another whole, which makes it the biggest advantage of the device and distinguishes it from many other inventions. In addition, based on work on augmented reality technology and virtual reality that face problems due to weight and size (such as lenses, glasses, computers and smart phones), it is feasible to create a single, integrated device that contains all elements of AR technology and works perfectly and artistically without any weight and size issues.

[0038] According to the present invention, the GPS navigation device 112 provides an initial route towards the desired destination as input by the user. The initial route is determined by collecting latitude and longitude coordinate data while driving, which is then passed to the CPU 106 of the smart glass 102. The CPU 106 sends the received latitude and longitude coordinate data to a data center (preferably controlled by a traffic control center), which is a collection of servers connected to each other and designed or programmed to act based on the received data. The data center includes a comprehensive geographic database 104, which includes the required updated information related to the roads in the area of ​​interest. The database 104 also enables the augmented reality content to be associated with a specific geographic location and simultaneously projected onto the smart glass 102. No additional initiation (such as scanning a QR code or image) is required from the user side to facilitate the projection of the augmented reality content onto the smart glass 102. In another embodiment of the present invention, the central database is managed by an entity or any road management agency or organization used for road management, which regularly updates the central database with new road signs or warnings as per international standards for roads and always on existing or new roads that meet public safety and traffic standards.

[0039] Additionally, multiple cameras 108 located at various locations on the vehicle and on the smart glass 102 constantly monitor the surroundings during the process of navigating through a determined route (provided by a GPS navigation device). These cameras, driven by 3D imaging technology including infrared and RGB specifications with depth sensing capabilities, also work in conjunction with motion sensors.

[0040] FIG2( a) shows a conventional windshield without the smart glass according to the present invention, and FIG2( b) shows how objects are displayed on the smart glass according to the present invention. The driver of a vehicle using a windshield without smart glass views the external environment as depicted in FIG2(A). In both scenarios, the road is free of any road facilities. However, according to the present invention, and as shown in FIG2( b), all necessary and required signs or obstacles to be viewed by the driver are displayed as 3D objects on the smart glass 202. The driver sees road signs 204 and possible temporary obstacles 206 as 3D objects displayed on the smart glass 202, thereby enabling the road to be free of any physical signs or obstacles, and resulting in an intelligent and visually attractive road or highway, and enhancing the safety of its users.

[0041] While driving, cameras of the surrounding environment work with motion sensors using depth sensors and infrared imaging specifications to collect depth information and 3D data. 3D objects (upcoming road signs, road obstacles, emergency warnings or direction signs) to be displayed on the smart glass at a specific distance appear on the smart glass with a hologram effect. Multiple cameras detect the surrounding environment, and a geographic database that works in combination with the cameras sends information related to the specific 3D object to be displayed on the smart glass at a specific point in time. Multiple cameras use light reflected from motion sensors to determine the exact target distance. The time it takes for light or light pulses to transition back and forth from the camera to a specific object ensures that the displayed 3D object is placed as it would be seen in real space.

[0042] RGB cameras work with the usual ambient lighting to simultaneously interpret and then specify the required 3D objects on the smart glass based on the detected scenery or surroundings. These cameras use regular images, providing depth information and distance to the sensor using each pixel. In addition, instead of performing a recording function, the camera works as a medium to assist augmented reality technology, and thus displays the elements in the best possible way through a combination of data transmission together with integrated optics and holograms. This is due to light and acceleration sensors, which measure the lighting of the natural environment and match the measured lighting when illuminating default objects so that these objects appear more natural, and also by calculating the accuracy of the distance before displaying the object as a hologram.

[0043] The analog signal from the reflected infrared signal is converted into a digital format by combining the 2D image with the obtained position information, and then the 2D image is processed to be displayed as a 3D object or hologram on the smart glass. However, with the help of a CPU with a powerful processing capability that supports a high-power graphics processor to engage in augmented reality technology with very complex specifications, the displayed 3D object floating on the smart glass will not distract the driver or cause discomfort such as nausea. Through regular updates of the central database, newly added elements on the road are constantly added and integrated so that they are reflected on the smart glass at the right time and place while driving. This has nothing to do with vehicle speed, position, or distance between objects.

[0044] In addition to saving a lot of money for the agencies implementing road projects for the implementation of smart and agile roads, the implementation of smart glass for vehicles can also be used as a new source of investment and national revenue. The enhanced road space can be rented out to competing companies for advertising and other commercial purposes in order to achieve a large economic return with limited material costs. Considering that smart glass helps in reducing environmental pollution by eliminating the need for street furniture materials and the associated manufacturing stages (including smoke generation, transportation, installation and removal procedures), the proposed invention is also most beneficial to the surrounding environment.

[0045] In another embodiment of the present invention, multiple light and acceleration sensors are also used in the proposed system to measure the illumination values ​​from the natural surrounding environment, and then the measured illumination values ​​are matched when illuminating the objects projected on the smart glass. This feature makes the displayed 3D objects look natural. Moreover, the accurate calculation of the distance between the positions of the objects to be displayed enables the objects to be perceived as being naturally present. Moreover, to enhance the experience, the visibility of the displayed objects increases as the vehicle approaches, and the visibility decreases as the vehicle moves away from the position where the object is displayed.

[0046] In another embodiment of the present invention, the smart glass is automatically activated when the vehicle containing the smart glass is started, and because it is mechanically and electrically connected, it is an integral part of any car. In a sense, when the vehicle engine is started and prepared for driving mode, because the smart car glass device is directly connected to the vehicle and cannot be controlled separately, the smart car glass device automatically works and does not need to be operated by the driver or user. The reason is that the smart car glass device is a centrally operated device that does not allow any role of the user to control the input and output. In another embodiment of the present invention, when the car is working and the smart car windshield is started, the operating system of the car immediately starts to operate, and the operating system of the car is designed and developed for the purpose of achieving all results of the tasks of the device by working with augmented reality technology and linking to the internal and external components of the device in the best way, which in turn also work directly when the system works. When driving starts, the main result is to start the smart glass work in all its parts and inside and outside the system. The strategic concept of the smart glass car of the present invention is consistent with the future strategic concept of global industrial products, which is oriented towards artificial intelligence to achieve higher productivity without significant consumption of resources, and save a lot of money in road projects and maintenance by reducing the maximum number of various works of road facilities and their manufacturing, installation, transportation and maintenance.

[0047] The purpose of the smart glass device for cars of the present invention is to enter a new era of beautiful and smart ways, and with the help of augmented reality technology, start to apply it in practice. Among them, usually car companies and factories and investment companies use their limited goals, which are to improve the specifications of cars technically to keep up with the pace of technological development, and enter the field of artificial intelligence and the marketing of their products, and continue to compete in the market with other companies and factories. But in this invention, we will achieve all the above-mentioned goals of car companies, while realizing the essence and main requirements of the idea, which starts a new stage - smart and beautiful roads using artificial intelligence by completely dispensing with all guiding and guiding road facilities, and creating a new environment for the road, while the driver will not feel the difference, and drive the vehicle in a completely safe way, just like these added elements are really real and are in the right place and the right time for the user when driving. In addition to saving billions of dollars for the agencies implementing the road projects, they can achieve very large economic returns with limited material costs by renting the enhanced road space to competing companies and using it for advertising and commercial purposes, making smart and agile roads by implementing the smart glass for cars of the present invention as a source of investment and national income.

[0048] After considering this specification and the accompanying drawings which disclose the preferred embodiments of the present invention, many changes, modifications, variations and other uses and applications of the present invention will become apparent to those skilled in the art. All such changes, modifications, variations and other uses and applications which do not depart from the spirit and scope of the present invention are deemed to be covered by the present invention, which is limited only by the appended claims.

Claims

1. A system for displaying information to a user of a vehicle, the system comprising: Smart glass for displaying the information as three-dimensional (3D) elements; Navigation assistance components; as well as Central database, Multiple sensor components, including position sensors, light sensors, and accelerometers; as well as A plurality of cameras located at multiple locations on the vehicle and on the smart glass for achieving complete coverage of the surrounding environment; wherein the smart glass and navigation aid components communicate with a central database that stores said information and facilitates sending and displaying said information on the smart glass in real time depending on the location through which the vehicle is being maneuvered, and Therein, light sensors and acceleration sensors measure illuminance values ​​from the surrounding environment and then match the measured illuminance values ​​when illuminating elements displayed on the smart glass.

2. The system according to claim 1, wherein: The information displayed may include road signs, obstacles or advertisements that are relevant to the location through which the vehicle is being maneuvered.

3. The system according to claim 1, wherein: The information is displayed on the smart glass using artificial intelligence (AI) technology and augmented reality (AR) technology.

4. The system according to claim 1, wherein: The information is displayed on the smart glass using marker-less augmented reality (AR) technology, thereby eliminating the need for an external device or projector for displaying the information.

5. The system according to claim 1, wherein: The navigation aid component is a Global Positioning System (GPS) component.

6. The system according to claim 1, wherein: Displaying relevant and updated signals in real-time on smart glass eliminates the need for physical signage on the road.

7. The system according to claim 1, wherein: The plurality of cameras have infrared (IR) imaging capabilities.

8. The system according to claim 1, wherein: The central database is managed by an entity which regularly updates the central database with new road signs or warnings.

9. The system of claim 1 further comprising a central processing unit (CPU) programmed to: process data received from the navigation assistance component and the plurality of cameras; and generate and display relevant information on the smart glass connected to a central database.

10. The system according to claim 9, wherein: The central processing unit supports a graphics processor.

11. The system of claim 1 , further comprising a three-dimensional digital compass.

12. The system of claim 1, wherein: When a vehicle incorporating the smart glass is started, the smart glass is automatically activated without having to be individually activated by the driver or user of the smart glass.

13. The system of claim 1, wherein: The information is displayed on the smart glass in real time.

14. The system of claim 1, wherein: Smart glass is designed to support three-dimensional vision, covering the driver or user's central field of view.

15. The system of claim 1, wherein: The smart glass is manufactured as a single piece so as not to affect the driver or user's field of view through the smart glass.

Images