System and method

Abstract

In accordance with the present disclosure, there is provided a system for displaying information to a user of a vehicle using augmented reality (AR), the system comprising a processor configured to: interact with a display unit for displaying information using AR in a field of view associated with the system and a data source for providing data to the processor; processing an AR image for display on the display unit according to a fit to the data, the AR image including an application spatial and temporal variation with respect to the fit to the data; and causing the display unit to display the AR image.

Description

Technical Field

[0001] The present invention relates to a system for displaying information to a user of a transportation vehicle using augmented reality (AR), a head-up display (HUD) system, a method for displaying information using AR, and a computer, a computer program, and a non-transitory computer-readable storage medium. Background Art

[0002] Known heads-up display (HUD) systems used in vehicles such as automobiles and aircraft provide a vehicle user with computer-generated virtual images that augment the user's visual sensory perception of real-world features or objects visible during vehicle operation. The HUD system can be configured to generate virtual images and project the virtual images as images or text through a windshield or other display screen so that the user can view information while looking up without diverting attention from real-world features visible during vehicle operation.

[0003] In known systems, an AR image may be displayed to a user to indicate a planned or intended route of a vehicle. In a mode of vehicle operation where the user controls the vehicle, such an AR image may enable the user to operate the vehicle to follow the route. In a mode of vehicle operation where the user does not control the vehicle (e.g., an autonomous vehicle mode), such an AR image may enable the user to monitor the planned maneuvers of the vehicle before executing them.

[0004] There is usually a degree of uncertainty associated with the planned or expected route of the vehicle or the location where the AR image should be displayed. This can cause the displayed AR image to not match the environment that the AR image is being displayed to enhance. For example, it may be desirable to position the AR image on the road surface in front of the vehicle, but due to inaccuracies in the calculations, the AR image may be displayed as being raised above the road surface. This can create a "parallax effect" that causes problems for users of the vehicle.

[0005] Acknowledging this, if the user believes that the AR image does not match the environment with a high degree of accuracy, the user may tend to ignore the AR image. This ultimately leads to the user's distrust of the system. In this case, the AR image displayed to the user causes an unnecessary distraction to the user of the vehicle. This is a problem for general use of the system and a comfortable fit with the system, and it also poses a significant safety issue in terms of distracting the driver's attention or causing the driver to be too focused on the AR function.

[0006] It is an object of the present invention to provide an improved system and / or method thereof and / or to address one or more of the problems discussed above or elsewhere, or to at least provide an alternative system and / or method. Summary of the invention

[0007] The following overview is about some aspects. These aspects are only aspects of the present invention, wherein the systems and methods are as defined in the appended claims. The reader will understand that features of aspects that do not fall within the scope of the present invention may still be included in aspects of the present invention that do fall within the scope of the present invention. Therefore, any aspects lacking these features are reserved to provide the reader with useful background information.

[0008] A first aspect provides a system for displaying information to a user of a vehicle using augmented reality (AR), the system comprising a processor configured to: interact with a display unit for displaying information using AR in a field of view associated with the system and a data source for providing data to the processor; process an AR image for display on the display unit based on a fit with the data, the AR image comprising an applied spatial and temporal change relative to the fit with the data; and cause the display unit to display the AR image.

[0009] In one example, the processor is configured to generate the AR image including the applied spatial and temporal changes relative to the fit to the data.

[0010] In one example, the processor is configured to receive the AR image, the AR image comprising the applied spatial and temporal changes relative to the fit to the data.

[0011] In one example, the applied spatial and temporal variations are based on a noise function.

[0012] In one example, the fit to the data is a navigation fit for enabling the AR image to be optionally displayed as a navigation aid, wherein the fit to the data is optionally an expected travel path of the vehicle, wherein the fit to the data is based on a route spline of waypoints and / or a lane marking or lane boundary.

[0013] In one example, the AR image defines a 2D cross section and / or a 3D volume; and / or a 2D cross section and / or a 3D volume is defined by or relative to the fit.

[0014] In one example, the AR image optionally includes a first AR image portion representing the fit to the data and a second AR image portion including the applied spatial and temporal changes, wherein the first AR image portion has a first size and the second AR image portion has a second size that is smaller than the first size.

[0015] In one example, the applied spatial and temporal variations are provided in a first dimension and / or a second dimension.

[0016] In one example, the applied spatial and temporal variations vary within a boundary optionally defined by or relative to the fit, or vary within a two-dimensional cross-section and / or a three-dimensional volume defined by or relative to the fit.

[0017] In one example, the processor is configured to process an AR image having a first characteristic under a first vehicle motion parameter and to process an AR image having a second characteristic under a second vehicle motion parameter.

[0018] In one example, the processor is configured to display the AR image with an offset relative to a feature that the AR image is intended to enhance.

[0019] A second aspect provides a head-up display (HUD) system, comprising the system according to the first aspect.

[0020] A third aspect provides a means of transport comprising a system or a HUD system according to the first aspect or the second aspect.

[0021] A fourth aspect provides a method for displaying information to a user of a vehicle using augmented reality (AR), the method comprising: interacting with a display unit for displaying information using AR in a field of view and a data source for providing data to the processor; processing an AR image for display on the display unit based on a fit with the data, the AR image including an applied spatial and temporal change relative to the fit with the data; and causing the display unit to display the AR image.

[0022] A fifth aspect provides a computer comprising a processor and a memory configured to implement the method according to the fourth aspect.

[0023] A sixth aspect provides a computer program comprising instructions which, when executed by a computer comprising a processor and a memory, cause the computer to implement the method according to the fourth aspect.

[0024] A seventh aspect provides a non-transitory computer-readable storage medium comprising instructions that, when executed by a computer comprising a processor and a memory, cause the computer to implement the method according to the fourth aspect.

[0025] Of course, it should be understood that the features described in relation to one aspect of the present invention may be included in other aspects of the present invention. For example, the method of any aspect of the present invention may include any features described in relation to the device of any aspect of the present invention, and vice versa.

[0026] Other preferred and advantageous features of the invention will become apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings, in which:

[0028] Figure 1.1 A schematic diagram showing a system for displaying information to a user of a transportation vehicle using augmented reality (AR);

[0029] Figure 1.2 Display the field of view of the user and the display unit;

[0030] Figure 2 A schematic diagram showing a head-up display (HUD) system;

[0031] Figure 3 Displaying a first view, wherein the AR image is displayed;

[0032] Figure 4 show Figure 3 A magnified view of an area of

[0033] Figure 5 displaying a second view in which the AR image is displayed;

[0034] Figure 6 show Figure 5 A magnified view of an area of

[0035] Figure 7 displaying a third view in which the AR image is displayed;

[0036] Figure 8 show Figure 7 A magnified view of an area of

[0037] Fig. 9 displaying a fourth view in which the AR image is displayed;

[0038] Fig.10 displaying a fifth view in which the AR image is displayed;

[0039] Fig.11 Display including system or HUD system of the vehicle; and

[0040] Fig.12 Shows general methodological principles. DETAILED DESCRIPTION

[0041] The following description describes some embodiments. These embodiments are merely embodiments of the present invention, wherein the systems and methods are as defined in the subsequent claims. The reader will understand that features of embodiments that do not fall within the scope of the present invention may still be included in embodiments of the present invention that do fall within the scope of the present invention. Therefore, the description of any embodiment lacking these features is reserved to provide the reader with useful background information.

[0042] refer to Figure 1.1 , showing a system 10 for displaying information to a user of a means of transportation using augmented reality (AR). The system 10 includes a processor 100. The processor 100 is configured to interact with a display unit 200 and a data source 300. The processor 100 being configured to interact with the display unit 200 and the data source 300 may mean that the processor 100 is configured to receive information from the display unit 200 and the data source 300 and to transmit information to the display unit 200 and the data source 300. Specifically, the data source 300 is configured to provide data to the processor 100. The processor 100 is configured to receive data from the data source 300 accordingly.

[0043] Here, the "user of the transport" is typically the driver or operator of the transport. However, the user may be one or more other users of the transport, for example, a passenger. In one example, the transport may be operated in a fully autonomous mode (including a mode without any direct control from the user of the transport (e.g., the driver)). In this case, the driver may indeed be a passenger in the transport. In fully autonomous mode, despite not controlling the transport himself, the system 10 (and / or the HUD system 50 described below) can enhance the field of view by displaying AR images on the display unit 200 to encourage the user to keep his attention on the road ahead. This can be called "gamification" of the display. Even more, if the transport is not driven or controlled, games can be played while using the transport. Games may involve the use of AR images.

[0044] For the avoidance of doubt, the system 10 can be modified in accordance with a vehicle comprising the display unit 200 and the data source 300. In this case, the system 10 itself need not comprise the display unit 200 and the data source 300. However, in an alternative configuration not involving modification, the system 10 may include the display unit 200 and the data source 300. This may be a physical modification or a software update or upgrade.

[0045] The display unit 200 is used to display information in a field of view (FoV) associated with the system 10 using AR. The display unit 200 may include a holographic display unit or may be in the form of a holographic display unit. A holographic display is a display that uses light diffraction to produce a virtual three-dimensional image. The holographic display unit may include a combiner. The combiner may be a windshield (i.e., all or a portion of a windshield). That is, the system 10 is an optical see-through system. An optical see-through system allows a user to "directly" view the real world. These AR systems add virtual content in the form of AR images by adding additional light to the light from the real world.

[0046] refer to Figure 1.2 , showing a plan view of an exemplary FoV of a user and a FoV associated with the system 10. The user's FoV (referred to herein as the "user FoV") is indicated at 20, and the FoV associated with the system 10 is indicated at 30.

[0047] In this example, the FoV associated with the system 10 is the FoV of the display unit 200. As shown, the user's FoV is greater than the FoV of the display unit 200. Typically, the user's FoV is in the range of 120 to 200 degrees. The FoV of the display unit has a vertical FoV of approximately 4 degrees and a horizontal FoV in the range of 15 to 20 degrees. As can be understood from this figure, the display unit 200 uses AR to display information in the user's FoV.

[0048] In the description herein, "the FoV of the display unit" means the FoV of the augmentable area. This is the portion of the human field of view in which the display unit 200 can display virtual content to augment the real-world scene. That is, the FoV of the display unit 200 is the area that is viewable by the user and in which the display unit 200 can augment the user's real-world scene. In this sense, the FoV of the display unit can be referred to as the "field of view of the user using the system 10" or the "field of view of the user using / viewing the display unit 200." In a HUD system (e.g., the HUD system 50), this can be referred to as the "field of view of the user using the HUD system 50."

[0049] In the following description, the FoV of the display unit 200 will be simply referred to as “FoV”.

[0050] It can be understood that the display unit 200 is located in an area that can be viewed by the user. In a specific example, this means that the display unit 200 is located at or in front of the windshield (that is, facing the user), and the user views the road ahead through the windshield when using the transportation vehicle. In other examples, the display unit 200 can be set on the side or rear of the transportation vehicle, so the FoV can be located on one side behind the transportation vehicle.

[0051] In the figure, the user's FoV and the FoV of the display unit 200 are shown as rectangular FoVs. It can be understood that, in reality, such FoVs are not rectangular.

[0052] As described above, the data source 300 is used to provide data to the processor 100. The processor 100 is configured to display information to the user at the display unit 200. The information provided to the user is based on the data provided by the data source 300. The information provided to the user may represent the data provided by the data source 300. The processing of the AR image displayed to the user is based on the fit with the data. The processed AR image itself may represent the fit with the data. Alternatively, the processing may be based on the fit with the data, and an appropriate AR image may be selected accordingly. The processor 100 may determine the fit with the data. The fit with the data may provide a balance position for the AR image or for a portion or image component of the AR image.

[0053] Data source 300 may be a system, component or may be a data storage. For example, data source 300 may be a global positioning system (GPS), a mapping system / service or other navigation system. Alternatively, data source 300 may be a component such as a detector or a memory. The detector may be a camera, a radar and / or a lidar. Alternatively, data source 300 may be a data storage such as a data table. Data source 300 may include one or more systems, components or data storages.

[0054] The data provided to the processor 100 may be GPS data, navigation data, route data and / or feature data. The navigation data may be based on the Navigation Data Standard (NDS).

[0055] GPS data may include a set of geographic coordinates received from a mapping service. Navigation data and route data may include data points received from a navigation or set-course system. Navigation data and route data may include a set of data points associated with an intended path of a vehicle.

[0056] The feature data may be related to features detected in the environment outside the vehicle. The feature may include another vehicle, for example, a road vehicle or an off-road vehicle. The feature may also include road signs or general road facilities. The feature may be a person or an animal. The feature may include the layout of a road or the layout of road lanes. The feature data may be related to all of these features. The features to which the feature data relates may vary depending on the use or mode or type of system, vehicle or scenario.

[0057] Here, the “environment outside the vehicle” may include features in front of the vehicle, on the side of the vehicle, and behind the vehicle. In other words, the feature data may be related to features outside the FoV.

[0058] The processor 100 is configured to process an AR image for display on the display unit 200. An AR image is an image displayed using augmented reality (AR). The processor 100 is configured to process an AR image in a number of different ways as will be described in further detail below.

[0059] The processor 100 is configured to cause the display unit 200 to display an AR image.

[0060] Although the processor 100 causes the display unit 200 to display an AR image may be implemented in actual implementation, it may be considered optional for some embodiments. For example, the core distinguishing feature is processing the AR image based on the fit to the data, and the AR image includes applied spatial and temporal changes relative to the fit to the data. In terms of distinguishing the key features of the present invention, causing the display unit 200 to display an AR image may be optional.

[0061] Of course, those skilled in the art may understand how an AR image may be displayed on the display unit 200 , and the above provides an illustrative example of how an AR image may be displayed on the display unit 200 .

[0062] refer to Figure 2 , a head-up display (HUD) system 50 is shown. The HUD system 50 includes a picture generation unit 500, a combiner unit 600, and a corrector unit 700.

[0063] Combiners and correctors may be used, for example, in holographic systems or other systems that present a see-through surface to a user but in which image features are provided by the system. A combiner may be, for example, an at least partially (e.g., semi) transparent surface used in a system to overlay an image presented by a projector (which may be part of or separate from the image generating unit 500) onto the user's physical world. The combiner is at least partially transparent and allows the user to see through while reflecting or otherwise directing (e.g., AR functionality) light to the user. Corrector units may be used to correct aberrations, filters, and / or improve light utilization efficiency, or generally correct some optical property. In some examples, one or more optical devices or lenses may be used that include filters.

[0064] The system 10 described herein may be incorporated into a HUD system 50. The system 10 and / or the HUD system 50 may be incorporated into or form part of an advanced driver assistance system (ADAS).

[0065] Although the present disclosure is related to the system 10 and HUD system 50 for installation in a vehicle, the present disclosure can also be applied in the context of a head-mounted display (HMD). For example, the system 10 can be incorporated into a pair of smart glasses or other wearable devices.

[0066] Figures 3 to 10 Each shows a front view through a vehicle windshield, with an AR image displayed in the FoV. The operation and functionality of the system 10 will be described with reference to these figures. In each figure, the AR image includes applied spatial and temporal variations relative to the fit to the data. The spatial and temporal variations may be referred to as "uncertainty components." Spatial and temporal variations may "distort" the AR image.

[0067] In one embodiment, the processor 100 is configured to generate an AR image including applied spatial and temporal changes relative to the fit to the data. That is, the processor 100 itself generates the AR image. In this way, additional components for generating the AR image may not be required.

[0068] In another embodiment, the processor 100 is configured to receive an AR image including applied spatial and temporal changes relative to the fit to the data. That is, the processor 100 itself does not need to generate the AR image. This can reduce the computational load on the processor 100 while allowing dedicated components to generate the AR image.

[0069] refer to Figure 3 , showing a first view, in which an AR image 3110 is displayed. 3000 indicates the user's FoV. 3100 indicates the FoV (that is, the FoV of the display unit 200). 3200 indicates an area of ​​the FoV, which is enlarged and displayed on Figure 4 middle.

[0070] The processor is configured to process the AR image 3110 for display on the display unit 200 according to the fit with the data from the data source 300 .

[0071] The AR image 3110 includes applied spatial and temporal changes. Applied spatial and temporal changes are changes in space and time relative to the fit to the data. This can also be referred to as spatial and temporal deviations from the fit to the data.

[0072] In this way, when the AR image 3110 is displayed, any uncertainty or difference with respect to the features it is intended to enhance is not perceived. In addition, the user can understand that there is a certain degree of uncertainty in the positioning of the AR image 3110 based on spatial and temporal changes. This solves the problem of user interaction and experience with the system 10. User perception is improved because the spatially and temporally varying AR images improve the user's perception of the AR images at different angles, distances, and vehicle speeds. This also alleviates the problem of the "strobing effect" effect of the AR image.

[0073] Furthermore, it may be particularly difficult to accurately place (i.e., actually on) an AR image, for example, on a road surface. This may be due to inaccurate data related to the road surface height. Although the positioning of the AR image 3110 is inaccurate, applying spatial and temporal changes makes any inaccuracies imperceptible and improves the appearance of the AR image, ultimately improving the user experience of the system 10 and increasing vehicle safety because the user is not distracted by an imperfectly matching AR image 3110.

[0074] It is understood that in conventional art methods, AR images are usually identical to or closely mapped to a fitted visual representation of the data. Therefore, it is very counterintuitive to actually do essentially the opposite - apply spatial and temporal changes. However, the opposite approach solves the problems of conventional art methods, or at least provides a viable alternative to existing methods.

[0075] Spatial variation is a variation in space relative to the fit to the data. That is, spatial variation may be a variation in space relative to the fit to the data. As can be seen from the figure, the AR image 3110 may include image portions that vary in space toward and away from the fit to the data.

[0076] Temporal variation is the change in time relative to the fit to the data. That is, the change is not a fixed spatial change from the fit, but rather varies over time.

[0077] In this example, the spatial and temporal variations are applied according to a noise function. That is, the spatial and temporal variations are not periodic or repetitive variations. More specifically, the spatial and temporal variations may appear to be random (e.g., uncertain or probabilistic). Such variations may be referred to as "lightning bolt appearance", "lightning bolt variation", or "lightning bolt manner", or may take the form of "lightning", "lightning variation", or "lightning manner". This improves the perception of AR images at different angles, distances, and vehicle speeds because any difference between the ideal position of the AR image and its actual displayed position is not perceived. In addition, such AR images can produce AR effects close to objects, road features, or expected paths of travel.

[0078] refer to Figure 4 , showing a magnified view of the area of ​​the FoV. As shown, the fitting with the data is a navigation fitting, which is used to enable the AR image 3110 to be displayed as a navigation aid. Here, the data provided by the data source 300 includes geographic coordinates received from a mapping service, which can be GPS data, navigation data and / or route data. The geographic coordinates can be or represent waypoints.

[0079] In this case, the fit to the data is the expected path of travel of the vehicle. An example of such a fit is a "route smoothing curve". The route smoothing curve can be based on waypoints. Due to the inherent inaccuracy of the data provided by the data source 300, the route smoothing curve may have unrealistic, unintended and / or unusual characteristics, such as zigzag characteristics or a jittery appearance. If displayed alone, this will negatively affect the perception of the AR image.

[0080] The AR image 3110 includes a first AR image portion 3112 and a second AR image portion 3114. The first AR image portion 3112 represents a fit to the data. The second AR image portion 3114 includes applied spatial and temporal changes. Figures 4 to 6 The fit used is a smooth curve, but for example as follows about Figures 7 to 10 The alternative fit may be based on coordinates representing lane markings, lane boundaries, or other road features. The spatial and temporal variation of the second AR image portion 3114 may be provided to be close to the first AR image portion 3112. That is, the second AR image portion 3114 may vary near the first AR image portion 3112. The first AR image portion 3112 may provide a varying equilibrium position.

[0081] In this way, the first AR image portion 3112 represents the actual fit to the data, while the second AR image portion 3114 is provided to mask or improve the perception of any uncertainty in the fit to the data.

[0082] The first AR image portion 3112 has a first size. The first size is the width or height of the "line" of the first AR image portion 3112. In the case where the first AR image portion has a circular cross-section, the first size can be related to the diameter of the line. The second AR image portion 3114 has a second size. The second size is the width or height of each "line" of the second AR image portion 3114. In the case where the lines of the second AR image portion 3114 have a circular cross-section, the second size can be related to the diameter of each line.

[0083] The second size is smaller than the first size. In this case, when the size is a diameter, the diameter of each line of the second AR image portion 3114 is smaller than the diameter of the line of the first AR image portion 3112 .

[0084] In this way, the route smoothing curve is represented as a first portion 3112 having a first larger size and a second portion of a second smaller size, wherein the first portion represents the fit to the data and the second portion represents the "uncertainty" component associated with the data. The user can therefore take advantage of the fit to the data (i.e., the route smoothing curve) and use the route smoothing curve as a navigation aid without being confused by any errors in its position relative to the road surface, because the second AR image portion 3114 makes such errors imperceptible.

[0085] Specifically, the AR image 3110 may include a first AR image portion 3112 and a second AR image portion 3114, wherein the first AR image portion 3112 includes a single main line, and the second AR image portion 3114 includes three supplementary lines. As described above, the three lines of the second AR image portion 3114 present spatial and temporal changes relative to the first AR image portion 3112.

[0086] The AR image 3110 defines a two-dimensional cross section and / or a three-dimensional volume. In this way, the curvature of the AR image 3110 can be better presented to the user.

[0087] In one example, the two-dimensional cross section and / or the three-dimensional volume are defined by or relative to the fit. The AR image 3110 can vary within the two-dimensional cross section and / or the three-dimensional volume. The cross section or volume may be related to the degree of uncertainty of the data. In this way, the user can interpret an AR image containing a low degree or low amount of spatial and temporal variation as representing accurate data, and interpret an AR image containing a high degree or high amount of spatial and temporal variation as representing inaccurate data. It is understood that this cross section or volume is not necessarily the cross section or volume of the displayed AR image. Alternatively, the AR image may be present within this cross section or volume.

[0088] Applying spatial and temporal variations is provided in the first dimension and / or the second dimension. The first dimension is the width. The second dimension is the height. Figure 3 and Figure 4 In the example, the application space and time variations are provided primarily in the first dimension, and also to a lesser extent in the second dimension. Advantageously, this effectively communicates the uncertainty of the navigation fit (e.g., the smooth curve of the route) to the user. This also effectively makes the uncertainty of the position of the AR image 3110 relative to the road surface invisible.

[0089] The degree of spatial and temporal variation in the first and second dimensions may be different. Figure 3 and Figure 4As shown, the spatial variation in width is greater than the spatial variation in height. This is helpful in cases where the AR image 3110 is used to provide navigation assistance (e.g., a smooth curve of the route in the figure). Alternatively, the spatial variation in height can be greater than the spatial variation in width. This is very helpful in cases where the AR image is used to indicate road signs or road features (e.g., curbs or lane markings). This may also be more special to the user, thereby increasing user engagement. An example of this is shown in the following description Figures 7 to 9 middle.

[0090] Different degrees of spatial and temporal variation in the first and second dimensions can be obtained by applying two different noise functions for use in each dimension to the AR image. The noise functions can each have different parameters that define the variation. For example, a first noise function can be applied to a first axis (e.g., the x-axis) and a second noise function can be applied to a second axis (e.g., the y-axis).

[0091] In one example, the application of spatial and temporal variation varies within a boundary. That is, the boundary may define a maximum degree of variation or the second AR image portion 3114. The second AR image portion 3114 may vary within the boundary (possibly randomly and / or according to a noise function). For the avoidance of doubt, the boundary is not a feature displayed to the user, but rather an area that displays spatial and temporal variation in the AR image.

[0092] Boundaries may be defined by or relative to the fit. Alternatively or additionally, spatial and temporal changes may occur within a two-dimensional cross-section and / or three-dimensional volume defined by or relative to the fit. For example, boundaries may be set near the fit as an equilibrium position. In this way, the changes represent the fit to the data and enable the user to intuitively understand the position of the fit to the data while still reducing any parallax effects caused by inaccurate positioning of the AR image to augment the real world scene.

[0093] The second AR image portion 3114 may include a plurality of lines based on the first AR image portion 3112 or the same as the first AR image portion 3112 but with a different noise seed. In this way, the second AR image portion 3114 is related to the first AR image portion 3112 and to the fit of the data, and presents spatial and temporal variations.

[0094] refer to Figure 5 , a second view is displayed, in which an AR image 5110 is displayed.

[0095] 5000 represents the user's FoV. 5100 represents the FoV (i.e., the FoV of the display unit 200). 5200 represents a region of the FoV, which is enlarged and displayed on the Figure 5 middle.

[0096] about Figure 3 and Figure 4 The description of the processing of the AR image provided is also the same as the description Figure 5 and Figure 6 5110 is related to the AR image 5110 displayed in the display unit 200. Specifically, the processor is configured to process the AR image 5110 for display on the display unit 200 according to the fit with the data from the data source 300. The AR image 5110 includes applied spatial and temporal changes. The applied spatial and temporal changes are changes in space and time relative to the fit with the data. This can also be called a spatial and temporal deviation from the fit with the data. The AR image 5110 includes a first AR image portion 5112 and a second AR image portion 5114.

[0097] From about Figure 3 and Figure 4 The described AR image 3110 understands other features of the AR image 5110 .

[0098] Reference now Figures 3 to 6 , the AR image is selected or adjusted based on the vehicle motion parameters. For example, based on the vehicle motion parameters, the AR image can be made narrower, wider, darker, or brighter. In one advantageous embodiment, the spatial and temporal changes in the AR image narrow as the vehicle speed increases and widen / broaden as the vehicle speed decreases. In this way, the AR image can be less distracting to the user at high speeds, while the AR image can be clearer or more obvious to the user at low speeds. Furthermore, it is likely that more complex routes will be followed when driving at low speeds (e.g., city driving), so more visible AR images can provide clear and useful information to the user. Less complex routes are followed when driving at high speeds (e.g., highway driving), where route information is less important for a long period of time, so less visible AR images can reduce user distraction.

[0099] When the transport is operated according to the first transport motion parameter, the AR image has a first characteristic, and when the transport is operated according to the second transport motion parameter, the AR image has a second characteristic. When the transport is operated according to the first transport motion parameter, the spatial and temporal changes may have the first characteristic, and when the transport is operated according to the second transport motion parameter, the spatial and temporal changes may have the second characteristic. The transport motion parameter may include at least one of: speed, acceleration, time / distance to another transport. Adjusting the characteristics of the AR image according to the transport motion parameter may be referred to as a "shrink" function.

[0100] Figure 3 and Figure 4 The AR image 3110 shown has Figure 5 and Figure 6 The AR images shown in the figure show different spatial and temporal changes. Specifically, Figure 3 and Figure 4 The second AR image portion 3114 is shown to be narrower in width than the second AR image portion 5114 . Figure 3 and Figure 4 The speed of the transport vehicle is greater than Figure 5 and Figure 6 The speed of transport in .

[0101] The processor 100 is configured to display the AR image with an offset relative to the feature that the AR image is intended to enhance. The offset relative to the feature that the AR image is intended to enhance may be intentionally introduced. Nevertheless, the presence of spatial and temporal variation in the AR image mitigates any problems associated with the offset from the intended feature. This is because the spatial and temporal variation makes the offset imperceptible, and the user can still intuitively understand the intent of the AR image. In one example, the feature that the AR image is intended to enhance may be a road surface or lane marking. The AR image may be offset from such a feature by appearing to be "raised" from the road surface.

[0102] from Figures 3 to 6 It will be appreciated that the AR images 3110, 5110 are aligned with the position of the vehicle. The AR images 3110, 5110 extend away from the front of the vehicle. When the vehicle adjusts its position in the real world, the "shift" function can be used to reposition the AR images 3110, 5110 so that the AR images 3110, 5110 are realigned or repositioned with the vehicle. This is advantageous in the case of using standard definition mapping data, which has lower accuracy / quality than high-definition mapping data. The shift function handles the lower quality of standard definition map data by ensuring that the AR images 3110, 5110 remain aligned with the vehicle and are therefore visible to the user. This improves the user experience and perception of the system 10.

[0103] refer to Figure 7 , a third view is displayed, in which an AR image 7110 is displayed.

[0104] 7000 indicates the user's FoV. 7100 indicates the FoV (i.e., the FoV of the display unit 200). 7200 indicates an area of ​​the FoV, which is enlarged and displayed on the Figure 8 middle.

[0105] about Figures 3 to 6 The description of the processing of the AR image provided is also the same as the description Figure 7 and Figure 8 7110 is related to the AR image 7110 displayed in the display unit 200. Specifically, the processor is configured to process the AR image 7110 for display on the display unit 200 according to the fit with the data from the data source 300. The AR image 7110 includes applied spatial and temporal changes. The applied spatial and temporal changes are changes in space and time relative to the fit with the data. This can also be called a spatial and temporal deviation from the fit with the data. The AR image 7110 includes a first AR image portion 7112 and a second AR image portion 7114.

[0106] From about Figures 3 to 6 The described AR images 3110 , 5110 understand other features of the AR image 7110 .

[0107] Some features are Figure 7 and Figure 8 These features are unique to the AR image 7110. These features will be described in this article.

[0108] The data provided by data source 300 includes data related to lane boundaries. The data may be GPS data of geographic coordinates related to the location of the lane boundaries. Alternatively or additionally, the data may be feature data related to lane boundaries or lane markings. For example, such feature data may be provided by a detector.

[0109] AR image 7110 indicates the location of the lane boundary.

[0110] Applying spatial and temporal variations is provided in the first dimension and / or the second dimension. The first dimension is the width. The second dimension is the height. Figure 7 and Figure 8 , the application space and time variations are provided primarily in the second dimension, and also to a lesser extent in the first dimension. Advantageously, this draws the user's attention to the presence of lane boundaries. Furthermore, this effectively communicates the uncertainty of the navigation fit (e.g., the smooth curve of the route) to the user. This also effectively makes the uncertainty of the position of the AR image 7110 relative to the road surface invisible.

[0111] Different degrees of spatial and temporal variation in the first and second dimensions can be obtained by applying two different noise functions for use in each dimension to the AR image. The noise functions can each have different parameters that define the variation. For example, a first noise function can be applied to a first axis (e.g., the x-axis) and a second noise function can be applied to a second axis (e.g., the y-axis). The second noise function can have parameters that allow a greater variation on the y-axis than the variation on the x-axis allowed by the first noise function.

[0112] The degree of spatial and temporal variation may change as the vehicle moves closer to the lane boundary. For example, the size of the second AR image portion 7114 may increase as the vehicle moves closer to the lane boundary. Furthermore, the size of the two-dimensional cross-section of one or more lines forming the first and / or second AR image portions 7112, 7114 may increase as the vehicle moves closer to the lane boundary. This helps to remind the driver to move the vehicle toward the adjacent lane. Ultimately, this improves road safety.

[0113] refer to Fig. 9 , a fourth view is displayed, in which an AR image 9110 is displayed.

[0114] The FoV (ie, the FoV of the display unit 200 ) is represented by 9100.

[0115] about Figure 3 and Figure 8 The description of the processing of the AR image provided is also the same as the description Fig. 9 9110 is related to the AR image 9110 displayed in the display unit 200. Specifically, the processor is configured to process the AR image 9110 for display on the display unit 200 according to the fit with the data from the data source 300. The AR image 9110 includes applied spatial and temporal changes. The applied spatial and temporal changes are changes in space and time relative to the fit with the data. This can also be referred to as a spatial and temporal deviation from the fit with the data. The AR image 9110 includes a first AR image portion 9112 and a second AR image portion 9114.

[0116] From about Figures 3 to 8 The described AR images 3110 , 5110 , 7110 understand other features of the AR image 9110 .

[0117] Some features are Fig. 9 Specific to the AR image 9110. These features will be described in this article.

[0118] The spatial and temporal variation of the AR image 9110 can vary according to the audio signal. The audio signal may be related to or be music played in the sound system of the vehicle. For example, an increase in the level of the audio signal will result in an increase in spatial and temporal variation. This can improve the user experience by matching the signal to the audio signal. In addition, this can offset any distraction to the user caused by the increase in the volume of the audio signal by making the AR image 9110 more prominent in the FoV.

[0119] refer to Fig.10 , a fifth view is displayed, in which an AR image 10110 is displayed.

[0120] The user FoV is represented by 10000. The FoV (ie, the FoV of the display unit 200) is represented by 10100.

[0121] about Figures 3 to 9 The description of the processing of the AR image provided is also the same as the description Fig.10 10110 is related to the AR image 10110 displayed in the display unit 200. Specifically, the processor is configured to process the AR image 10110 for display on the display unit 200 according to the fit with the data from the data source 300. The AR image 10110 includes applied spatial and temporal changes. The applied spatial and temporal changes are changes in space and time relative to the fit with the data. This can also be referred to as a spatial and temporal deviation from the fit with the data. The AR image 10110 includes a first AR image portion 10112 and a second AR image portion 10114.

[0122] From about Figures 3 to 9 The described AR images 3110 , 5110 , 7110 , 9110 understand other features of the AR image 10110 .

[0123] Some features are Fig.10 These features are unique to the AR image 10110. These features will be described in this article.

[0124] The AR image 10110 provides the user with information related to the road turn. The AR image 10110 is provided as a series or group of arrows. That is, the AR image 10110 can be provided to enhance features other than the road surface, lane boundaries, or lane markings.

[0125] The AR image 10110 may be aligned with the curb. The AR image 10110 may be provided aligned with the curb and vertically above the curb.

[0126] The spatial and temporal variations make any uncertainty in the positioning of the AR image 10110 relative to background features (including curbs) imperceptible. In this way, the user experience is improved. Furthermore, the spatial and temporal variations can provide a consistent user interface by matching the overall appearance of the user interface. This can be called "electric skin".

[0127] refer to Fig.11 , showing a vehicle 8000. The vehicle 8000 includes the system 10 or the HUD system 50.

[0128] refer to Fig.12, showing a method for displaying information to a user of a means of transport using augmented reality (AR). Step S9010 includes interacting with a display unit for displaying information using AR in a field of view and a data source for providing data to a processor. Step S9020 includes processing an AR image for display on the display unit based on a fit to the data, the AR image including applied spatial and temporal changes relative to the fit to the data. Step S9030 includes causing the display unit to display the AR image.

[0129] Although step S9030 may be implemented in actual implementation, it can be considered optional for some embodiments. For example, the core distinguishing feature is the processing step. In terms of the key features that distinguish the present invention, the actual display step may be optional.

[0130] While preferred embodiments have been shown and described, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the appended claims and described above.

[0131] At least some of the example embodiments described herein may be constructed in part or in whole using dedicated hardware. Terms such as "component", "module" or "unit" as used herein may include, but are not limited to, hardware devices, such as circuits in the form of discrete or integrated components, field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs), which perform certain tasks or provide related functions. In some embodiments, the described elements may be configured to exist on a tangible, persistent, addressable storage medium and may be configured to execute on one or more processors. In some embodiments, these functional elements may include, for example, components, such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, routines, code snippets, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Although example embodiments have been described with respect to components, modules and units discussed herein, such functional elements may be combined into fewer elements or separated into additional elements. Various combinations of optional features have been described herein, and it is understood that the described features may be combined in any suitable combination. In particular, the features of any one exemplary embodiment may be combined with the features of any other embodiment as appropriate, unless such combinations are mutually exclusive. Throughout the specification, the term "comprising" means including specified components but does not exclude the presence of other components.

[0132] Attention is paid to all papers and documents which are filed concurrently or prior to the specification related to this application and which are open to public inspection with the present specification, and the contents of all such papers and documents are hereby incorporated herein by reference.

[0133] All features disclosed in this specification (including any accompanying claims, abstract and drawings) and / or all steps of any method or process disclosed as such may be combined in any combination, except combinations in which at least some of these features and / or steps are mutually exclusive.

[0134] Unless otherwise expressly stated, each feature disclosed in this specification (including any attached claims, abstracts and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose. Therefore, unless otherwise expressly stated, each feature disclosed is only an example of a series of general equivalent or similar features.

[0135] The present invention is not limited to the details of the foregoing embodiments. The present invention extends to any novel feature or any novel combination of features disclosed in this specification (including any attached claims, abstract and drawings), or to any novel method or any novel combination of steps of any method or process disclosed in this specification (including any attached claims, abstract and drawings).

Claims

1. A system for displaying information to a user of a transportation vehicle using augmented reality (AR), the system comprising a processor configured to: interacting with a display unit for displaying information using AR in a field of view associated with the system and a data source for providing data to the processor; processing an AR image for display on the display unit according to a fit to the data, the AR image comprising applying spatial and temporal changes relative to the fit to the data; and The display unit is caused to display the AR image.

2. The system according to claim 1, characterized in that The processor is configured to generate the AR image, the AR image comprising the applied spatial and temporal changes relative to the fit to the data.

3. The system according to claim 1 or 2, characterized in that: The processor is configured to receive the AR image, the AR image comprising the applied spatial and temporal changes relative to the fit to the data.

4. A system as claimed in any one of the preceding claims, characterised in that The applied spatial and temporal variations are based on a noise function.

5. A system as claimed in any one of the preceding claims, characterised in that The fit to the data is a navigation fit, for enabling the AR image to be optionally displayed as a navigation aid, wherein the fit to the data is optionally an expected travel path of the vehicle, wherein the fit to the data is a route smooth curve based on waypoints and / or a lane marking or lane boundary.

6. A system as claimed in any one of the preceding claims, characterised in that The AR image defines a 2D cross section and / or a 3D volume; and / or a 2D cross section and / or a 3D volume is defined by or relative to the fit.

7. A system as claimed in any one of the preceding claims, characterised in that The AR image optionally includes a first AR image portion representing the fit with the data and a second AR image portion including the applied spatial and temporal changes, wherein the first AR image portion has a first size and the second AR image portion has a second size that is smaller than the first size.

8. A system as claimed in any one of the preceding claims, characterised in that The applied spatial and temporal variations are provided in a first dimension and / or a second dimension.

9. A system as claimed in any one of the preceding claims, characterised in that The applied spatial and temporal variations vary within a boundary optionally defined by or relative to the fit, or vary within a two-dimensional cross-section and / or a three-dimensional volume defined by or relative to the fit.

10. A system as claimed in any one of the preceding claims, characterised in that The processor is configured to process an AR image having a first characteristic under a first vehicle motion parameter and to process an AR image having a second characteristic under a second vehicle motion parameter.

11. A system as claimed in any one of the preceding claims, characterised in that The processor is configured to display the AR image with an offset relative to a feature that the AR image is intended to enhance.

12. A heads-up display (HUD) system comprising a system as claimed in any preceding claim.

13. A vehicle comprising a system or a HUD system as claimed in any preceding claim.

14. A method for displaying information to a user of a transportation vehicle using augmented reality (AR), the method comprising: interacting with a display unit for displaying information in a field of view using AR and a data source for providing data to the processor; processing an AR image for display on the display unit according to a fit to the data, the AR image comprising applying spatial and temporal changes relative to the fit to the data; and The display unit is caused to display the AR image.

15. A computer comprising: A processor and a memory configured to implement the method of claim 14; a computer program comprising instructions that, when executed by a computer comprising a processor and a memory, cause the computer to implement the method of claim 14; or a non-transitory computer-readable storage medium comprising instructions that, when executed by a computer comprising a processor and a memory, cause the computer to implement the method of claim 14.

Images