Context sensitive overlay of content via augmented reality devices
By using geolocation technology and wireless communication infrastructure of mobile communication devices in augmented reality devices, the context-sensitive content related to real objects is directly transmitted, solving the problem of delay in augmented reality devices in the prior art and improving the user experience.
Patent Information
- Application Number
- CN202380076949.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-11-02
- Filing Date
- 2023-10-24
- Publication Date
- 2025-06-13
AI Technical Summary
Existing augmented reality devices may introduce unacceptable delays when presenting associated virtual features to individuals, affecting the user experience.
By utilizing geolocation technology of mobile communication devices, the location of augmented reality devices is estimated and context-sensitive content related to real objects is transmitted through wireless communication infrastructure, reducing dependence on image capture and key point extraction, thereby reducing latency.
Reduced latency when presenting associated content to individuals, improving the performance and user experience of augmented reality devices.
Smart Images

Figure CN120153340A_ABST
Abstract
Description
BACKGROUND OF THE DISCLOSURE 1. TECHNICAL FIELD
[0001] The present disclosure generally relates to augmented reality devices, as such devices can be used to present context-sensitive content to an individual.
[0002] 2. Information
[0003] The World Wide Web (or simply the Web), enabled by Internet computing, switching, and wireless and wired transmission resources, has grown rapidly in recent years. This growth has facilitated a wide variety of different types of transactions using a wireless Internet infrastructure. This allows an individual to remain connected or coupled to the Internet while driving a vehicle (e.g., an automobile or motorcycle), walking, cycling, or engaging in various other activities. The recently established wireless Internet infrastructure has facilitated this connectivity, which allows an individual to shop, perform electronic banking, stay in touch with friends, family, work colleagues, business partners, etc. while being active in a highly mobile society. This allows an individual to go about their daily routines of working, going to school, shopping, driving, etc. without losing connectivity for extended periods of time.
[0004] In some instances, such as when an individual is sitting in a vehicle, walking, or cycling in an unfamiliar environment, the individual may benefit from visual cues to assist in obtaining, for example, automotive services (e.g., fuel, charging), food and beverages, etc. Thus, an individual may utilize a communication device such as a mobile cellular communication device, which can notify the individual as to the availability of desired goods and / or services. In a particular example, certain types of augmented reality devices can present an overlay of symbols that an individual can view in addition to an image of real objects within an area of interest. Thus, it is understood that providing useful information relevant to an individual's interests and needs remains an active area of research. SUMMARY OF THE INVENTION
[0005] One general aspect includes an augmented reality controller for transmitting content to an augmented reality device in communication with a vehicle. The augmented reality controller has one or more processors communicatively coupled to at least one memory device including computer code. The at least one memory device and the computer code are configured to use the one or more processors to estimate the position of the augmented reality device relative to one or more known regions of a grid. The one or more processors of the augmented reality controller are further configured to determine the orientation of the field of view of the augmented reality device and transmit context-sensitive content via the vehicle to the augmented reality device, the context-sensitive content being related to or otherwise associated with one or more real objects visible within the field of view of the augmented reality device, wherein the content is at least partially generated based on the one or more known regions of the grid.
[0006] In certain embodiments, an augmented reality controller having one or more processors coupled to at least one memory device additionally accesses a cloud-based data storage device to determine relevant context-sensitive content based at least in part on one or more content preferences of an individual currently in the vehicle or meeting a predefined proximity condition relative to the vehicle. In certain embodiments, an augmented reality controller having one or more processors coupled to at least one memory device additionally directs access to content from a data storage device at least in part based on a user's selection of a travel type context from a plurality of selectable travel type contexts. In certain embodiments, an augmented reality controller having one or more processors (the one or more processors being coupled to at least one memory device) additionally transmits context-sensitive content related to the one or more real objects via the vehicle to an augmented reality device in response to a selection of a travel type, at least in part based on the positioning of the one or more real objects along a driving travel path of the vehicle. In certain embodiments, an augmented reality controller having one or more processors additionally transmits context-sensitive content associated with the one or more real objects via the vehicle to an augmented reality device in response to a selection of a travel type, at least in part based on the positioning of the one or more real objects along a walking travel path. In certain embodiments, an augmented reality controller having one or more processors coupled to at least one memory device additionally receives a selection of at least one known region of a grid from one or more known regions in a plurality of uniformly sized regions proximate an estimated position of a neighboring vehicle. In certain embodiments, the plurality of uniformly sized regions correspond to regions of approximately 3.0 m by approximately 3.0 m. In certain embodiments, an augmented reality controller having one or more processors coupled to at least one memory device additionally determines whether one or more real objects visible within a field of view of the augmented reality device are relatively close or relatively far from an individual co-located with the augmented reality device. In certain embodiments, an augmented reality controller having one or more processors coupled to at least one memory device additionally determines whether one or more real objects visible within a field of view of the augmented reality device are located within a peripheral portion or a central portion of a field of view of an individual co-located with the augmented reality device. The central portion may correspond to the direct line of sight of the individual. In certain embodiments, an augmented reality controller having one or more processors coupled to at least one memory device additionally receives from a user interface of the vehicle a selection to display context-sensitive content using alphanumeric characters, graphical icons, or a combination thereof.
[0007] Another general aspect includes a method of providing content to an augmented reality device in communication with a vehicle, the method including parsing the position of the augmented reality device relative to one or more known regions of a grid. The method further includes determining the orientation of the field of view of the augmented reality device. The method further includes generating and / or transmitting via the vehicle context-sensitive content associated with one or more real-world objects visible within the field of view of the augmented reality device, at least in part based on the one or more known regions of the grid.
[0008] In certain embodiments, the method may further include directing access to a data storage device to determine associated context-sensitive content at least in part based on one or more content preferences of an individual co-located with or proximate to the vehicle. In certain embodiments, the method may further include selecting a travel type context of the individual from a plurality of travel type contexts. In certain embodiments, the method may further include transmitting context-sensitive content associated with the one or more real-world objects in response to selecting the travel type, at least in part based on the positioning of the one or more real-world objects along the driving travel path of the vehicle. In certain embodiments, the method may further include selecting one or more known regions of the grid proximate to the position of the augmented reality device and / or the vehicle from a plurality of uniformly sized regions. In certain embodiments, these uniformly sized regions correspond to regions of approximately 3.0 m by approximately 3.0 m.
[0009] Another general aspect includes a non-transitory computer-readable medium including program instructions for the following actions: causing an augmented reality device controller to perform an estimation of the position of the augmented reality device relative to one or more known regions of a grid. The non-transitory computer-readable medium may further cause the augmented reality device controller to perform a determination of the orientation of the field of view of the augmented reality device. The non-transitory computer-readable medium may further cause the augmented reality device controller to transmit context-sensitive content via the vehicle to the augmented reality device, at least in part based on the one or more known regions of the grid, the context-sensitive content being associated with one or more real-world objects visible within the field of view of the augmented reality device.
[0010] In certain embodiments, the non-transitory computer-readable medium may also cause the augmented reality device controller to direct access to a cloud-based data storage device to determine associated context-sensitive content based at least in part on one or more content preferences of an individual co-located with the augmented reality device and / or the vehicle. In certain embodiments, the non-transitory computer-readable medium may also cause the augmented reality device controller to perform a selection of a travel type context of an individual from a plurality of travel type contexts. In certain embodiments, the non-transitory computer-readable medium may also cause the augmented reality device controller to perform a determination as to whether one or more real-world objects visible within the field of view are relatively close or relatively far from an individual co-located with the augmented reality device. In certain embodiments, the non-transitory computer-readable medium may also cause the augmented reality device controller to perform a determination as to whether one or more real-world objects visible within the field of view of the augmented reality device are located within the periphery or directly in the line of sight of an individual co-located with the augmented reality device. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The claimed subject matter is particularly pointed out and distinctly claimed in the concluding portion of the specification. However, both as to its organization and / or method of operation, features and / or advantages thereof, may best be understood by reference to the following detailed description in conjunction with the drawings in which:
[0012] Figure 1 is a diagram of a vehicle having a wireless connection to a communication infrastructure and coupled to augmented reality glasses, according to an embodiment.
[0013] Figure 2 is a diagram of a vehicle wirelessly communicating with augmented reality glasses to display associated content generated by a content generator, according to an embodiment.
[0014] Figure 3 depicts a view of a passenger located in a vehicle and viewing real-world objects while traveling on a road, according to one or more embodiments.
[0015] Figure 4 depicts an individual wearing augmented reality glasses while walking in an urban environment, according to an embodiment.
[0016] Figure 5 depicts a dashboard of a vehicle, according to an embodiment, showing a display for configuring the augmented reality glasses.
[0017] Figure 6 is a diagram showing, according to an embodiment, Figure 2 a content generator and a portion of a city, the portion of the city having been divided into uniformly sized grid regions.
[0018] Figure 7is a diagram showing real objects visible through augmented reality glasses according to an embodiment, where some real objects are near the augmented reality glasses and other real objects are relatively far from the augmented reality glasses.
[0019] Figure 8 is a diagram showing certain angles in the line-of-sight region and the peripheral region visible through augmented reality glasses according to an embodiment.
[0020] Figure 9 is a first flowchart of a method for context-sensitive overlay of content via an augmented reality device according to an embodiment.
[0021] Figure 10 is a second flowchart of a method for context-sensitive overlay of content via an augmented reality device according to an embodiment.
[0022] Figure 11 is a diagram of a communication infrastructure including both wireless and wired communication devices and components according to various embodiments.
[0023] Figure 12 is a diagram showing a computing environment according to an embodiment.
[0024] In the following detailed description, reference is made to the accompanying drawings, which form a part of this description, where like reference numerals may refer to corresponding and / or similar like components throughout. It should be understood that the drawings are not necessarily drawn to scale for simplicity and / or clarity of illustration. For example, the dimensions of some aspects may be exaggerated relative to other aspects for ease of discussion, etc., and one or more aspects, features, etc. may be omitted. Further, it should be understood that other embodiments may be utilized. Additionally, structural and / or other changes may be made without departing from the claimed subject matter. References throughout this specification to "the claimed subject matter" refer to the subject matter intended to be covered by one or more claims or any part thereof, and do not necessarily refer to a complete set of claims, a particular combination of claims (e.g., method claims, apparatus claims, etc.), or a particular claim. Detailed Description
[0025] Throughout this specification, references to a particular embodiment, an embodiment, a specific embodiment, a particular implementation, etc. mean that the particular features, structures, characteristics, etc. described with respect to that particular embodiment and / or implementation are included in at least one particular embodiment and / or implementation of the claimed subject matter. Thus, the appearances of such phrases are not necessarily intended to refer to the same particular embodiment or implementation or to any one particular embodiment or implementation. Further, it should be understood that the particular features, structures, characteristics, etc. can be combined in various ways in one or more particular embodiments and / or implementations and are thus within the scope of the claimed subject matter. Generally speaking, for the specification of a patent application, these and other issues are likely to vary in particular usage scenarios. In other words, throughout this disclosure, the particular scenarios described and / or used provide guidance on the reasonable inferences to be drawn. The phrase "as the term is used herein", without further qualification, generally refers at least to the context of this patent application.
[0026] As previously mentioned, the World Wide Web, as enabled by Internet computing, switching, and wireless and wireline transmission resources, has grown rapidly in recent years. Such growth has facilitated a wide range of electronic communications using wireless Internet infrastructure. Thus, in many instances, an individual can remain connected or coupled to the Internet while driving a vehicle (e.g., an automobile or motorcycle), walking, running, cycling, or participating in various other activities. The recently established wireless Internet infrastructure has facilitated such connectivity, which allows individuals to shop, perform electronic banking, play games, and stay in touch with friends, family, work colleagues, business partners, etc. while being active in a highly mobile society. This allows individuals to carry out their daily routines of work, school, shopping, driving, etc. while staying connected with friends, family, colleagues, etc.
[0027] An important component that facilitates continuous connectivity of an individual to the global wireless communication infrastructure is the mobile cellular phone. As the capabilities of mobile cellular devices continue to increase, such devices are becoming increasingly useful as intermediaries between, for example, wearable devices and the overall wireless communication infrastructure. For example, smartwatches, biometric identification sensors, physiological sensors, etc. can operate in association with a mobile communication device of a single subscriber. In recent years, in addition to the devices identified above, augmented reality devices such as augmented reality glasses or other wearable augmented reality devices have become increasingly popular as a means of presenting visual cues to an individual, thereby allowing such individuals to more quickly gather information about their surroundings, receive visual warnings about potential hazardous events, and / or become more time-efficient in finding desired goods and / or services.
[0028] In one example, such as when an individual (e.g., a driver, a passenger) is sitting in a vehicle, walking, or cycling in an unfamiliar environment, the individual may benefit from visual cues to assist in obtaining certain services. Such services may include obtaining automotive fuel, locating an electric vehicle charging mechanism, obtaining food and beverages, obtaining directions to a residence or an establishment, etc. In certain situations, certain augmented reality wearable devices (such as augmented reality glasses) may present an overlay of virtual features (e.g., symbols), which the individual can view in addition to the real objects within the field of view of the augmented reality glasses.
[0029] However, certain wearable devices (such as certain types of augmented reality glasses) may introduce unacceptable latency when presenting associated virtual features (e.g., symbols and / or other parameters) to an individual wearing the wearable device. For example, some augmented reality device technologies utilize an imaging device (e.g., a camera), which may operate by capturing an image and uploading certain image parameters over a wireless communication link to a centralized image processing system. In one example, the imaging device may extract one or more key points from the captured image, which may involve computer vision techniques (e.g., image recognition). The parameters of the key points may be compared with images stored in a database until a match between the captured image and the stored image is determined. In response to such a match, the image processing computer may insert appropriate visual content or other content, which may be overlaid or otherwise positioned at a location determined or estimated relative to the one or more extracted key points. Thus, it can be appreciated that such processes involving image capture, key point extraction, transmitting appropriate parameters over a wireless infrastructure for comparison with images from a centralized database, subsequent detection of a match using the stored and captured image key points, and subsequent content insertion and transmission over the wireless infrastructure may consume several seconds or more. This latency reduces the attractiveness of wearable augmented reality devices (such as augmented reality glasses).
[0030] However, in certain non-limiting embodiments as described herein, relevant or associated content may be transmitted and displayed via a wearable augmented reality device (e.g., augmented reality glasses) without introducing an undesirable delay in presenting the relevant or associated content to an individual. For example, in a particular embodiment, a mobile communication device (e.g., a mobile phone) that may be held or carried by an individual wearing augmented reality glasses may use geolocation techniques (such as those described below) to estimate, resolve, or otherwise determine the location of the mobile communication device. The mobile communication device may transmit location information describing the determined, estimated, or resolved location to one or more processors of another computing device / system. The location information may include a small set of identifiers, such as location coordinates describing, for example, longitude, latitude, and altitude, and may be transmitted via a wireless communication infrastructure. In response to receiving the estimated location coordinates, the one or more computer processors may access a data storage device (also referred to as a data storage apparatus), which may provide appropriate visual content (also referred to as displayable content), such as, for example, symbols and / or alphanumeric text, that is relevant or otherwise associated with the determined, estimated, or resolved location. The visual content may then be transmitted via the wireless communication infrastructure and superimposed on the individual's augmented reality glasses or other wearable device. Thus, the superimposition of visual content for AR applications may occur with much less latency than competing methods, such as those involving extracting key points of a captured image and comparing the key points from the captured image to a key point database, either of which may introduce significant latency in generating content associated with an image viewed via augmented reality glasses).
[0031] In this context, the term "augmented reality device" can include any device that allows viewing of a real scene (also referred to as a physical scene) including one or more real objects (also referred to as one or more physical objects) through one or more screens (e.g., the transparent lenses of AR glasses) and also allows viewing of artificial objects (also referred to as virtual objects), which can be superimposed on the real scene on the screen (e.g., superimposed on one or more transparent lenses). Thus, an augmented reality (AR) device can include, for example, glasses that allow viewing of real objects in a scene augmented with artificial (e.g., computer-generated) content. Thus, in one example, a passenger or operator (e.g., a driver) of a transportation vehicle can use an AR device to view real objects in the vehicle environment (such as other vehicles, traffic signs, lane markings, etc.), as well as computer-generated visual features (e.g., symbols, text, etc.). In this example, the visual content presented by the AR device can identify, for example, real objects in the vehicle environment, as well as computer-generated text and / or icons, which can provide context and / or metadata that describes or otherwise relates to these real objects. The AR device in this example can be AR glasses. In another example, an augmented reality (AR) device can include a projector that displays an image on a vehicle windshield, where such an image identifies real objects that can be viewed through the windshield or provides context and / or metadata related to the real objects.
[0032] In some embodiments, an augmented reality device used by an individual may determine a descriptor or other information indicative of the location of the individual, such as by calculating the location of the individual using GPS or other satellite positioning signals. In certain embodiments, in addition to determining such location information, the augmented reality device (e.g., augmented reality glasses) may also determine and transmit information indicative of the orientation or heading of the field of view of the individual using the augmented reality device. More specifically, this may be the field of view visible through one or more lenses of the augmented reality device or visible through a vehicle windshield. Thus, in response to, for example, an individual wearing augmented reality glasses and the individual being located within a particular known grid area, the augmented reality device or another computing device may obtain or determine the known grid area that the individual is looking at based on, for example, the orientation angle of the field of view visible through the augmented reality glasses. Thus, in one possible example, in response to determining that an individual wearing augmented reality glasses is looking north when at a particular corner in an urban area, one or more computer processors communicatively coupled to a database may generate and transmit content related to or otherwise associated with one or more known areas of the grid immediately north of the location. Additionally, the one or more computer processors may determine the content based on the individual's gaze or more specifically based on the area to which the individual's gaze is directed. For example, the augmented reality device may determine that the individual's gaze is focused on a nearby area. In such an example, the one or more computer processors may provide content specifically related to or associated with the nearby area, at least in part based on the determination from the augmented reality glasses. Alternatively, in response to the augmented reality glasses determining that the individual's gaze is focused on an area relatively far from the individual, the one or more computer processors may provide content related to or associated with the one or more distant areas.
[0033] In certain embodiments, one or more computer processors coupled to a database may transmit or emit content that is consistent with a context selected for an individual wearing or otherwise using an augmented reality device, such as augmented reality glasses. As the term is used herein, "context" refers to the background, environment, and / or circumstances of the use of an augmented reality device, such as augmented reality glasses or other wearable augmented reality devices. Thus, in certain embodiments, such as those involving a driver or passenger of a vehicle, augmented reality glasses may, for example, be used in a vehicle "driver" context, in which (e.g., exclusively) safety-related content (e.g., characters and / or symbols and / or icons) is superimposed on real-world objects visible through the glasses. Similarly, in certain embodiments as previously indicated, augmented reality glasses may be used in a vehicle "passenger" context, which may result in the display of other alphanumeric messages, symbols, indicators, etc., which may have more informational content that may be consistent with the passenger's preferred content. In certain embodiments, a user interface, such as a menu displayed on a vehicle's dashboard, may present different types of content that may be selected by a driver, passenger, or other individual, such as content related to charging stations, food and beverage establishments, shopping, etc., and the content presented on the augmented reality device may be based on that selection. Additionally, the display mode via the augmented reality glasses, such as text descriptions and / or illustrative graphic icons, may also be selectable via the user interface. Further, also in certain embodiments, additional content may be presented to a user, such as in response to the user subscribing to enhanced augmented reality content. In one example, in response to the user approaching a food and beverage establishment, the augmented reality glasses may display the menu of the establishment, at least in part based on the user's paid subscription to such additional content. In another example, when approaching a museum, the augmented reality glasses may display a discount code, a brief description of the current display at the museum, opening / closing times, and / or admission fees (to name just a few non-limiting examples).
[0034] Additionally, in certain embodiments, the augmented reality device may detect an individual's current context as a "walking" context. This walking context may occur, for example, in response to a driver or passenger leaving the vehicle and walking around (possibly while the augmented reality device is within the vehicle's communication range). The augmented reality device may be a wearable device, such as the aforementioned augmented reality glasses. In the walking context, for example, the individual's wearable device may display a plurality of messages, symbols, indicators, etc., that may be appropriate for or identified as preferred by an individual walking along a street in a shopping district, entertainment district, restaurant district, or any other type of environment, and the claimed subject matter is not limited in this respect.
[0035] As used herein, "context-sensitive content" or similar terms refer to content provided to an individual based on the alternative background or environment of the individual (e.g., an individual driving a vehicle, a passenger in a vehicle, an individual walking within the communication range or other distance of the vehicle, etc.). Thus, for example, one or more computer processors in the individual's augmented reality device, the individual's vehicle, and / or a server communicating with the augmented reality device or the vehicle can detect the context in which the individual using the augmented reality device is the driver of a vehicle while, for example, the vehicle is traveling on a road. In response to such a determination, the one or more computer processors (which can be communicatively coupled to a database) can determine that in such a context, little or no content should be provided to the individual via the augmented reality device so as not to distract the individual while driving. In such a case, certain indicators can be provided to indicate that the vehicle fuel is relatively low or, for example, the battery power is low. In another example, the one or more processors can detect the context in which the individual using the augmented reality device is a passenger in a vehicle. In this context, the one or more computer processors (e.g., coupled to a database) can determine that the context-sensitive content for the individual relates to real-world objects such as landmarks, buildings, restaurants, refueling / charging stations, and / or other non-safety-related messages and / or symbols. In another example, when operating in a road travel context, augmented reality glasses can provide such context-sensitive content that provides notifications of upcoming potential emergencies, notifications related or associated with road conditions, notifications indicating upcoming traffic jams, construction areas, detours, etc.
[0036] In addition to providing context-sensitive content related or associated with real-world objects visible from a vehicle driving along a travel path, wearable augmented reality devices (such as augmented reality glasses) can provide content related to the walking travel path. In one possible scenario, when an individual is walking among establishments in a popular shopping district, the augmented reality glasses can indicate those stores and / or establishments offering discounts, promotions, and / or other offers. Additionally, in one example, the individual can select content preferences based on the individual's shopping preferences (e.g., women's clothing, men's clothing, sporting goods, kitchen utensils, etc.). At least partially based on such selected content preferences, when the individual is walking within various locations in the shopping district, the augmented reality device (e.g., augmented reality glasses) can display content that matches the selected content preferences.
[0037] It should be noted that an augmented reality device or another computing device can use signals from a satellite positioning system to estimate, resolve, or otherwise determine the location of an individual's mobile communication device in response to, for example, the individual walking among stores in an outdoor shopping mall area. However, in response to the individual walking among stores in an indoor shopping mall area, the mobile communication device can utilize other methods to determine, estimate, and / or resolve the location of the individual's mobile communication device.
[0038] According to the description of a general, overall communication infrastructure as shown and described in reference Figure 1 below, more specific implementations involving context-sensitive overlays of content via an augmented reality device are discussed. In Figure 1 one, for example, vehicle 102, which may represent an electric vehicle, may be equipped with an augmented reality controller 110 that is operable to format, generate, and / or transmit augmented reality content to an augmented reality device 145 of an operator (e.g., a driver) or a passenger of vehicle 102. Thus, in a particular implementation, as vehicle 102 travels along a travel path, an operator (and / or a passenger of vehicle 102) of vehicle 102 may view augmented reality content, for example, displayed by augmented reality device 145. As Figure 1 shown, the travel path of vehicle 102 may be divided into grid regions of substantially equal size (also referred to as grid areas). For example, as Figure 1 shown, vehicle 102 may cross a path that at least temporarily occupies grid region 105. Moments later, vehicle 102 may occupy, for example, an adjacent grid region. As vehicle 102 crosses a grid region, a computing device on the vehicle or another computing device may determine and report the location of vehicle 102, which may cause the display of content associated with or related to that grid region. In one possible example, when vehicle 102 crosses a region adjacent to a restaurant, an augmented reality device (e.g., augmented reality glasses 145) may display content associated with or related to the restaurant in response to an operator or passenger of vehicle 102 gazing in the direction of the restaurant. The content displayed may relate to the restaurant's menu, operating hours, specials, and so on.
[0039] The in-vehicle augmented reality controller 110 may facilitate the viewing of augmented reality content by a passenger and / or an operator (e.g., a driver) of vehicle 102, as Figure 1As shown. In certain embodiments, the augmented reality controller 110 can be a computing device (e.g., an infotainment system controller and / or a telematics control unit (TCU)) that forms or is part of a host unit that provides a human-machine interface (HMI), and can include a processor 115 and a memory device 120 (which can also be referred to as a non-transitory computer-readable medium). The memory device 120 can include instructions executable by the processor 115, including instructions for a context selection module 125, a position estimation module 130, and an AR content formatter module 135, which are discussed in more detail below. In these embodiments, the content visible to the passengers and / or operators of the vehicle 102 can be controlled by the processor 115 (e.g., including one or more computer processors), which can be coupled to the memory device 120 and can control functions associated with the augmented reality controller 110. In certain embodiments, the augmented reality controller 110 can include an input port that receives signals from a user interface (also referred to as an HMI), also known as user interface signals. In one embodiment, the user interface signals received by the controller 110 can relate to selectable content preferences expressed by the passengers and / or drivers of the vehicle 102. For example, in one embodiment, an operator or passenger of the vehicle 102 may prefer to specifically view safety-related content (e.g., safety-related text and / or icons or symbols) that can be related or associated with the grid region 105 or other grid regions traversed by the vehicle 102. In another example, when the vehicle 102 is traveling among grid regions (such as Figure 1 those shown), an operator or passenger of the vehicle 102 may prefer to view parameters related to upcoming points of interest. Additional specific implementation details regarding the selection by the passenger or operator of various contexts that can be selected via the context selection module 125 are described in more detail with respect to Figure 5 more detailed description.
[0040] The augmented reality controller 110 can additionally receive input signals corresponding to positioning signals, which can include positioning signals transmitted from a ground-based cellular transceiver (e.g., Figure 11 the cellular transceiver 1110) or can correspond to satellite positioning signals such as those provided by a satellite 1114 (also shown in Figure 11 ). In Figure 1 embodiments, such positioning signals can be used to assist the position estimation module 130 to facilitate the augmented reality content formatter 135 receiving signals representing appropriate content and formatting the content for delivery to the augmented reality glasses 145. The augmented reality controller 110 can include an input port for receiving content, for example, from a cloud-based content generator such as that described herein with respect to Figure 2 .
[0041] Vehicle 102 may include an embedded communication system capable of communicating with augmented reality glasses 145, or may utilize an individual's cellular mobile communication device to communicate with augmented reality glasses 145. The augmented reality glasses 145 may correspond to any one of a number of candidate augmented reality devices, which may include augmented reality helmets, augmented reality displays (e.g., head-up displays), augmented reality windshields, and the like. In one possible (and non-limiting) implementation, the augmented reality glasses 145 correspond to the Hololens system. Such augmented reality glasses include see-through holographic lenses, head tracking capabilities, inertial measurement capabilities using accelerometers, gyroscopes, at least one magnetometer, built-in spatial sound, and other sensing capabilities. It should be noted that the claimed subject matter is intended to cover the devices identified above and any and all augmented and / or mixed reality glasses, without limitation in fact.
[0042] Figure 2 is a diagram of a vehicle that wirelessly communicates with augmented reality glasses to display relevant or associated content generated by a content generator according to Embodiment 200. In Figure 2In an embodiment, the augmented reality glasses 145 may obtain positioning information from the vehicle 102 via a communication link directly with the vehicle 102 or via a communication link with one or more cellular communication devices that may be carried by a driver or passenger of the vehicle 102. Thus, the augmented reality glasses 145 may cooperate with the geolocation capabilities of the vehicle 102 to determine and / or report the position of the glasses 145 and the orientation of the field of view of the glasses 145. In a particular embodiment, the orientation of the field of view of the augmented reality glasses 145 may be determined via an output signal from at least one magnetometer or other type of sensor located within or at least coupled to the augmented reality glasses 145. Thus, the augmented reality glasses 145 may allow a driver, passenger, or other individual wearing the glasses and potentially located near or within the vehicle 102 to view real-world objects within the field of view of the augmented reality glasses 145 as well as computer-generated objects that may be displayed via the semi-transparent display of the glasses 145 and superimposed over the real-world objects. Thus, a driver, passenger, or other individual proximate to the vehicle 102 may be able to view real-world objects and view modifications or enhancements to the real-world objects via a layer of computer-generated objects or other features. Computer-generated features (e.g., computer-generated objects) may include symbols such as arrows, triangles, solid lines, dashed lines, and dotted lines, as well as any number of other shapes, which may include a variety of colors. Computer-generated objects may include alphanumeric characters, where in this context, alphanumeric characters refer to the characters of a standard alphabet (e.g., the 26 characters of the English alphabet, the 27 characters of the German alphabet, etc.) as well as the numeric characters 0 through 9. Thus, for example, an alphanumeric response may include a single letter character (such as "A"), multiple letter characters (such as "ABCDE"), a single or multiple numeric characters (such as "1" or "123"), and combinations thereof (such as "ABC123...", "A1B2C3..."). Additionally, the augmented reality glasses 145 may display computer-generated symbols such as graphical icons indicating the availability of food and / or beverage at a particular location, traffic conditions at a portion of a road, and any number of additional symbols, characters, and the claimed subject matter is not limited in this respect.
[0043] Accordingly, the augmented reality glasses 145 can display computer-augmented real objects to the driver or passengers of the vehicle 102. In a particular embodiment, in response to the driver of the vehicle 102 wearing the augmented reality glasses 145, a "driver" context can be selected or detected with or without receiving one or more input signals from, for example, the driver of the vehicle. In response to the selection or detection of the "driver" context, the AR controller 110 can cause the glasses 145 to exclusively display traffic and / or safety-related symbols and / or characters and avoid presenting potentially distracting content to the driver. In another embodiment, in response to a passenger of the vehicle 102 wearing the augmented reality glasses 145, a "passenger" context can be invoked or selected. In response to the selection or invocation of the "passenger" context, the AR controller 110 can cause the glasses 145 to display traffic and / or safety-related symbols and / or characters and can additionally display other symbols and / or characters, such as those related to the availability of various goods and services (such as food and drink, fuel and / or charging services, small rarities, etc.) in the area where the vehicle 102 is located.
[0044] Accordingly, it can be understood that the augmented reality glasses 145 can provide content that may be appropriate in any number of contexts, such as a travel type context, in the form of messages, symbols, indicators, etc. An individual's travel type context can indicate the travel mode in which the individual is engaged (e.g., whether the individual is traveling in a vehicle or on foot, or the type of vehicle in which the individual is traveling), and / or can indicate whether the individual has an operator role or a passenger role with respect to the vehicle. In a particular embodiment, the travel type context can include a driving context (and indicates that the individual is traveling in a vehicle as a driver), a passenger context (and indicates that the individual is traveling in a vehicle as a passenger), a walking context (and indicates that the individual is traveling by walking), or other contexts. Additionally, the travel type context can further describe one or more other circumstances or conditions of the individual's travel (also referred to as additional contexts or sub-contexts), such as the weather or road conditions of the travel. For example, the AR controller 110 or another computing device can detect a second, weather-related context that is part of the driving context, such as a context related to "rainy day" or "snowy day". In response to detecting such a weather-related context, the AR controller 110 can cause the AR glasses 145 to display additional safety-related computer-generated content, such as a message alert indicating that the driver may be traveling too fast for rainy or snowy road conditions or conditions related to reduced visibility. In certain embodiments, if the AR controller 110 or another computing device detects a driving context, the AR controller 110 can cause the AR glasses 145 to display symbols related to the driving direction, such as by displaying an arrow (e.g.) to indicate an upcoming left turn, right turn, stop sign, etc.
[0045] As previously described with respect to Figure 2As mentioned in the embodiments of, the augmented reality glasses 145 may perhaps cooperate with the vehicle 102 via a cellular or other type of wireless link between the glasses 145 and the vehicle 102. Such communication may provide information indicating the current position and orientation of the field of view 146 of the glasses 145 (see Figure 3 ). As further stated above, the augmented reality controller 110 may receive content from the content generator. Figure 2 An example content generator 220 is shown. Signals or other information representing the current position and field of view of the AR glasses 145 may be transmitted to the cellular transceiver 1110 and via the network 1130 (such as that referenced in Figure 11 ) to reach the content generator 220. In response to receiving an estimate of the position of the vehicle 102 and the orientation of the field of view of the glasses 145, the content generator 220 may operate to provide content related to or otherwise associated with real-world objects visible within the field of view of the glasses 145. In the Figure 2 embodiment, the content generator 220 includes a processor 222 (e.g., including one or more computer processors) that may communicate with a content module 224 to determine content suitable for transmission to the augmented reality glasses 145. The content suitable for transmission to the augmented reality glasses 145 may be determined in response to the processor 222 communicating with a data storage device 226 (also referred to as a data storage unit), which may include a memory device storing descriptors related to real-world objects visible at positions within the field of view of the glasses 145. The content generator 220 may then transmit the content, which is transmitted via the network 1130 and through the cellular transceiver 1110 for delivery to the vehicle 102 (or to the device of an individual co-located with or in the vicinity of the vehicle 102).
[0046] The content module 224 of the content generator 220 may operate in the cloud to generate content suitable for delivery to the augmented reality glasses 145 based on one or more input signals. For example, in Figure 2In an embodiment, content module 224 may provide content that is at least partially based on the current location of the augmented reality glasses 145, the selected type of travel context (e.g., driving context, passenger context, walking context, etc.), and the orientation of the field of view (FOV) of the glasses 145. In some embodiments, content generator 220 may select or otherwise generate content to be displayed to the driver and / or passenger at least partially based on the location of the vehicle, the user's preferences for desired content (e.g., nearby shopping, charging stations, food and beverage establishments), and the display mode (e.g., text and / or graphical icons), and the claimed subject matter is not limited in this regard. Thus, as a possible example, content generator 220 may receive a signal indicating that the augmented reality glasses 145 worn by the driver are currently located directly south of the entrance to Wrigley Field in Chicago, Illinois and the orientation of the field of view of the augmented reality glasses 145 is pointing north towards the entrance to Wrigley Field. In response to receiving such a signal, content module 224 may generate content indicating one or more symbols associated with the entrance to Wrigley Field. These computer-generated symbols may be layered or otherwise superimposed on real-world objects present in the scene within the field of view of the augmented reality glasses 145. In another possible example, in response to a signal indicating that the augmented reality glasses 145 worn by a passenger traveling within vehicle 102 are on an interstate highway in a desert portion of the United States where the geographic density of charging stations may be relatively low, content module 224 may generate content indicating the location and / or availability of vehicle charging services (such as a currently open fast charger located 5 km from the current location of vehicle 102 and adaptable to vehicle 102).
[0047] Figure 3 Depicts a view of a passenger in a vehicle looking at real-world objects in a scene in front of the vehicle while traveling on a road, according to embodiment 300. As Figure 3 shown, the passenger may view the scene through the windshield 210 and augmented reality glasses (with field of view 146). The real-world objects include various buildings, plants, road signs, etc. within the passenger's line of sight. The augmented reality glasses may display computer-generated content superimposed on the scene. The computer-generated content includes, for example, an alphanumeric message 325 that relates to the availability of a coffee shop along the vehicle's path and the approximate location of the coffee shop as indicated by arrow 330. Additionally, an alphanumeric message 310 is displayed to indicate the presence of a charging station. The augmented reality glasses 145 may also display computer-generated content indicating the approximate location of the charging station, such as arrows 315 and circles 320. As described above, some embodiments may provide content that is not limited to safety-related content if the augmented reality glasses 145 or another augmented reality device is being worn by a passenger rather than a driver. Thus, ifFigure 3 If the augmented reality glasses 145 in Figure 2 are currently being worn by a passenger of the vehicle 102, the content module 224 of Figure 3 can cause the augmented reality glasses 145 to display alphanumeric characters, symbols, and other content that does not necessarily relate to vehicle safety issues. Although not explicitly indicated in
[0048] Figure 4 depicts an individual wearing augmented reality glasses while walking in an urban environment. In such an embodiment (which may result from the individual 405 selecting the "walking" mode), Figure 2 the content generator 220 of Figure 4 can cause the augmented reality glasses 145 to display the presence of a restaurant (e.g., "Eatery") or a museum (e.g., as indicated by an appropriate icon) in the field of view 146 of the glasses 145. Although not explicitly indicated in Figure 4 when the individual 405 redirects the field of view of the glasses 145, different images and / or real objects may be present in the field of view of the glasses 145. In a particular embodiment, a user in the walking mode may, for example, choose to display only subscribed content. In this context, as
[0049] Figure 5Depicts the vehicle's dashboard 505, or more generally a user interface. According to embodiment 500, the user interface may display a menu that presents settings for configuring augmented reality glasses. The dashboard 505 may include gauges such as a speedometer, tachometer, fuel gauge (or charge indicator), and a display 510. In one embodiment, the display 510 may correspond to a configuration menu for an augmented reality display (such as an augmented reality display visible via the windshield 210 of the vehicle 102 (with a heads-up display) and / or via the augmented reality glasses 145). In Figure 5 an embodiment, the augmented reality content may be selected via, for example, a touchscreen display in which a passenger or driver of the vehicle 102 selects the specific content to be displayed. Such content may include points of interest (e.g., museums, parks, scenic viewpoints, historical interest items, etc.), charging stations (e.g., charging stations that provide a suitable charging waveform for the vehicle 102), food and beverage establishments (e.g., restaurants, coffee shops, drive-thru beverage establishments, etc.), shopping (e.g., department stores, hardware stores, etc.), and numerous other types of displayable augmented reality content. The display 510 may facilitate the selection of numerous additional configuration parameters for the augmented reality content, and the claimed subject matter is not limited in this regard. Additionally, the display 510 may also facilitate the operator or passenger of the vehicle 102 to select whether to display the augmented reality content via text-based descriptors (such as alphanumeric characters) and / or via graphical icons (such as icons representing coffee shops, charging stations, etc., as described with reference to Figure 3 . Thus, the display 510 may facilitate the selection of the type of augmented reality content to be displayed and whether to display such content using alphanumeric text, graphical icons, or a combination thereof.
[0050] Figure 6 is a diagram showing the Figure 2 content generator according to embodiment 600 and a portion of a city that has been divided into uniformly sized regions. As Figure 6 depicted, larger regions (such as regions that include one or more city blocks) may be divided into uniformly sized known grid regions. Within the known regions of the standard size of the grid, parameters related to the key points of real objects present in the uniformly sized regions may be loaded into a memory such as the data storage device 226. Thus, it can be understood that according to Figure 6, a computing device (e.g., the augmented reality controller 110 and / or the content generator 205) may track the position of an individual. When the individual moves from a first known uniformly sized region of the grid to a second known uniformly sized region of the grid, the computing device may obtain or otherwise generate content associated (e.g., related) with the second known uniformly sized region of the grid, and may cause the augmented reality glasses 145 to display the generated content. Additionally, the content may be determined to be relevant based on a travel type context (e.g., a road travel context, a walking travel context, etc.), where the travel type context may be determined based on a selection by the individual wearing the augmented reality glasses 145.
[0051] In Figure 6 an implementation, a computing device (e.g., the augmented reality controller 110 or the content generator 220 / 205) may obtain a selection indicating the position of an individual, such as an individual 405 manipulating a graphical user interface of a communication device (e.g., within a vehicle 102). This selection may be used to assist the content generator 205 / 220 in determining Figure 4 the exact position of the individual 405. For example, in response to the content generator 205 being unable to precisely determine, estimate, and / or resolve the position of the individual 405, the content generator 205 may transmit descriptors related to two or more uniformly sized regions that may be near the individual 405. The individual 405 may then be able to select the particular uniformly sized region in which the individual is currently located in response to performing a comparison between the descriptors of real objects received from a memory element (such as the data storage device 226) and real objects near or within the field of view of the individual 405. Thus, by presenting a particular image to the individual 405, the individual may assist in determining, estimating, and / or resolving his or her exact position relative to a particular uniformly sized region of a larger grid.
[0052] Figure 7 is a view showing real objects visible through the augmented reality glasses 145 according to an embodiment 700, where some real objects are considered to be near the augmented reality glasses or the individual wearing the augmented reality glasses, while other objects are considered to be far from the individual or the augmented reality glasses. If an object meets a predefined (or dynamically defined) proximity condition (e.g., a condition where these objects are within a predefined radius of the augmented reality glasses or the individual), then these objects may be considered to be near, and if these objects meet a predefined remote condition (e.g., a condition where these objects are outside a predefined radius of the augmented reality glasses or the individual), then these objects may be considered to be far. As Figure 7As shown, a region or area can be divided into regions of uniform size, such as a grid region measured to be approximately 3.0 meters by approximately 3.0 meters. However, it should be noted that alternative embodiments may include dividing a larger region into regions of non-uniform size. For example, in a particular embodiment, the area of a densely populated urban area can be divided into smaller-sized regions, such as regions measured to be approximately 2.0 meters by approximately 2.0 meters, regions measured to be approximately 3.0 meters by approximately 3.0 meters, or regions measured to be approximately 4.0 meters by approximately 4.0 meters, and the claimed subject matter is not limited to the specific dimensions of the smaller regions of uniform size of the larger region. Additionally, some portions of the larger region can be divided into smaller portions having a first area, while other portions of the larger region can be divided into smaller portions having a second area. For example, for an area transitioning from a densely populated urban area to a less densely populated suburban area, a first set of regions can be divided into smaller-sized regions, such as regions measured to be approximately 3.0 meters by approximately 3.0 meters. After transitioning from the urban area to the less densely populated suburban area, a second set of regions can be divided into regions measured, for example, to be approximately 4.0 meters by approximately 4.0 meters. Additionally, after transitioning from the suburban area to a rural area, a third set of regions can be divided into regions of different sizes, such as regions measured, for example, to be approximately 10.0 meters by approximately 10.0 meters. It should be noted that the claimed subject matter is intended to cover a wide variety of divisions of a larger region into smaller regions in order to provide an appropriate level of detail for the multiple real objects that may be expected to exist in a particular known area of the grid.
[0053] In Figure 7In an implementation, in response to a passenger in a vehicle selecting, for example, a road travel context, a computing device (e.g., the augmented reality controller 110 or the content generator 205 / 220) may cause the augmented reality glasses 145 to display content related to or associated with nearby real objects in addition to displaying content related to or associated with real objects that are relatively far from the augmented reality glasses 145. For example, the computing device may receive a user's selection that the user's context is as a passenger in a vehicle in a road travel context or that the user's context is a walking travel context. In response to such a user selection, the computing device may display an alphanumeric message to identify a nearby object 705. More specifically, the alphanumeric message may be a message that reads "Food Truck" and may be displayed within the field of view of the augmented reality glasses 145. Additionally, in response to an individual wearing the glasses lifting his or her head, the augmented reality glasses 145 may determine that the user is attempting to view a real object that is farther from the nearby object 705. Such a head movement (which may be accompanied by the detection of a head movement slightly to the left or right) may indicate that the individual wearing the glasses 145 is attempting to view a real object that is slightly farther from the individual. Accordingly, the augmented reality glasses 145 may display content related to a spa 710 that may be located at a position farther from the glasses 145 than the nearby object 705.
[0054] In one implementation, the augmented reality glasses may display the content based on whether the content falls within a central portion of the field of view of the augmented reality glasses or whether the content falls within a peripheral portion of the field of view. Figure 8 is a diagram showing certain angles that define a central portion and a peripheral portion of the field of view of the augmented reality glasses or another augmented reality device according to implementation 800. As Figure 8 indicated, a particular region and / or real object visible through the augmented reality glasses 145 may be considered to be in a more central, strictly direct line of sight of the field of view, such as falling within the Figure 8The region within the solid angle given by the predefined angle 01, where the angle 01 can be relative to the central axis of the field of view. The central part of the field of view can be a conical surface defined by the angle 01. On the other hand, other regions and / or real objects visible through the augmented reality glasses 145 can be considered to be more on the periphery of the field of view, such as the region outside the solid angle given by 01 up to the boundary of the angle given by 02. Thus, the peripheral part of the field of view can be the region outside the central part of the field of view but still within the conical surface defined by the angle 02. In a particular embodiment, the computing device (e.g., the augmented reality controller, the content generator, and / or the augmented reality glasses) can control how to display the content based on whether the content will fall within the central part of the field of view of the augmented reality glasses or whether the content will fall within the peripheral part. For example, the size of the content displayed on the augmented reality glasses can be based on whether the content falls within the central part (given by 01) or whether the content falls within the peripheral part (outside the given angle 01). For example, the delivered content for viewing within the solid angle given by 01 can be displayed using, for example, alphanumeric characters of a smaller size than the content for viewing outside the solid angle given by 01. For example, as Figure 8 shown, the text depicting "Content" (805) located within the angle given by 01 can be presented in a smaller text than the text depicting "Content" (810) located outside the solid angle given by 01. In a particular embodiment, presenting the content outside the more central part of the field of view can alert the individual to potential important visual cues that might otherwise not be noticed by an individual not wearing the augmented reality glasses 145.
[0055] Figure 9 is the first flowchart of a method for context-sensitive overlay of content via an augmented reality device according to embodiment 900. It should be noted that the processes represented by the Figure 9 flowchart and other flowcharts described herein can include actions other than those depicted in the figures, and / or can include actions arranged in an order different from the Figure 9 order shown. Figure 9 The method can start at 905, where the computing device (such as a vehicle (such as Figure 1A computing device (e.g., an augmented reality controller) on the vehicle 102 can prompt a user (which can correspond to the driver of the vehicle 102 or a passenger of the vehicle 102) to select a type of content (also referred to as a content type) for display. The content types for display can include, for example, general content, shopping-related content, content related to upcoming vehicle charging stations, upcoming vehicle fuel stations, content related to social media-driven events, content types related to banks and / or other financial-related institutions, and the like. In a particular embodiment, the operator of the vehicle 102 can select to view safety matters and / or upcoming vehicle charging stations in order to avoid being distracted by the displayed points of interest (which may interfere with attention and / or attentiveness). Also in a particular embodiment, a passenger of the vehicle 102 can select to view a wider variety of points of interest (POIs).
[0056] At 910, a computing device on the vehicle can, for example, receive a user's selection of a content type. The method can continue at 915, which can include the computing device on the vehicle prompting the operator to select, for example, POI content for display. Such content can include text and / or graphical icons for display as augmented reality content. In some embodiments, the displayed content can additionally include only subscription content, such as details of specific points of interest, food / drink establishments, charging station availability, and the like. The method can continue at 920, which can cause the vehicle 102 to communicate with the content generator 220 in order to filter out certain potentially distracting content and / or content that the operator or passenger of the vehicle 102 is less interested in. At 925, augmented reality content can be displayed to the user (e.g., the operator or passenger of the vehicle 102).
[0057] Figure 10 is a second flowchart of a method for context-sensitive overlay of content via an augmented reality device according to embodiment 1000. Figure 10 The method can begin at 1005, which can include parsing or estimating the position of the augmented reality device relative to one or more known regions of a grid (such as Figure 1 the grid region 105). The method can continue at 1010, which can include determining the orientation of the field of view of the augmented reality device (such as the augmented reality glasses 145). The method can continue at 1015, which can include generating and / or transmitting via the vehicle context-sensitive content associated with one or more real-world objects visible within the field of view of the augmented reality device based on one or more known regions of the grid.
[0058] Figure 11 is a diagram of a communication infrastructure including both wireless and wired communication devices and components according to various embodiments. In Figure 11In, corresponding to Implementation Example 1100, in addition to performing the motor transportation function, vehicle 102 provides the ability to communicate with a wireless communication infrastructure, either as an embedded ability or via a connection with any one of several types of mobile cellular communication devices such as a Bluetooth connection. Such abilities may include telephone communication, sending text messages, web browsing, providing a wireless hotspot ability, etc. In Figure 11 an implementation example, vehicle 102 may (either as an embedded ability or via a wired or wireless connection with an individual's mobile cellular communication device) transmit radio signals to a wireless communication network and receive radio signals from the wireless communication network. In one example, vehicle 102 may facilitate communication with a cellular communication network by transmitting wireless signals to and / or receiving wireless signals from a cellular transceiver 1110 over a wireless communication link 1123, and the cellular transceiver may include a wireless base transceiver subsystem, a Node B or an evolved Node B (eNodeB). Similarly, vehicle 102 may transmit wireless signals to and / or receive wireless signals from a local transceiver 1115 over a wireless communication link 1125. The local transceiver 1115 may include an access point (AP), a femtocell base station, a home base station, a small cell base station, a Home Node B (HNB) or a Home evolved Node B (HeNB), and may provide access to a wireless local area network (WLAN, e.g., an IEEE 802.11 network), a wireless personal area network (WPAN, e.g., a Bluetooth network) or a cellular network (e.g., an LTE network or other wireless wide area network, such as those discussed herein). Of course, it should be understood that these are only examples of networks that may communicate with a mobile device over a wireless link, and the claimed subject matter is not limited in this regard. In a particular implementation example, the cellular transceiver 1110, the local transceiver 1115, and the satellite 1114 represent contact points that allow vehicle 102 to interact with the network 1130.
[0059] In certain embodiments, the cellular transceiver 1110 and the local transceiver 1115 may communicate with the server 1140 via the communication link 1145, such as via the network 1130. Here, the network 1130 may include any combination of wired or wireless links and may include the cellular transceiver 1110 and / or the local transceiver 1115 and / or the server 1140. In a particular implementation, the network 1130 may include the Internet Protocol (IP) or other infrastructure capable of facilitating communication between the vehicle 102 at the call source and the server 1140 via the local transceiver 1115 or the cellular transceiver 1110. In one embodiment, the network 1130 may also facilitate communication between the vehicle 102 and the server 1140, for example, via the communication link 1160. In another implementation, the network 1130 may include cellular communication network infrastructure, such as, for example, a base station controller or a packet- or circuit-based switching center (not shown), to facilitate mobile cellular communication with the vehicle 102. In a particular implementation, the network 1130 may include local area network (LAN) elements such as WiFi APs, routers, and bridges, etc., and in such a case, may include a link to a gateway element providing access to a wide area network such as the Internet. In other implementations, the network 1130 may include a LAN and may or may not involve access to a wide area network, but may not provide any such access to the vehicle 102 (if supported). In some implementations, the network 1130 may include multiple networks (e.g., one or more wireless networks and / or the Internet). In one implementation, the network 1130 may include one or more serving gateways or packet data network gateways. Additionally, one or more servers 1140 may include an E-SMLC, a Secure User Plane Location (SUPL) Location Platform (SLP), a SUPL Location Center (SLC), a SUPL Position Center (SPC), a Position Determination Entity (PDE), and / or a Gateway Mobile Location Center (GMLC), each of which may be connected to one or more Location Retrieval Functions (LRFs) and / or Mobility Management Entities (MMEs) of the network 1130.
[0060] In certain embodiments, communication between vehicle 102 and a cellular transceiver 1110, satellite 1114, local transceiver 1115, etc. can occur using signals communicated across wireless or wired communication channels. Thus, the term "signal" can refer to communication using the propagation of electromagnetic or electronic signals via wired or wireless communication channels. Signals can be modulated to convey a message using one or more techniques such as amplitude modulation, frequency modulation, binary phase shift keying (BPSK), quadrature phase shift keying (QPSK), and numerous other modulation techniques, and the claimed subject matter is not limited in this respect. Thus, as used herein, the term "message" refers to a parameter, such as a binary signal state, that can be encoded in one or more signals using one or more of the modulation techniques identified above.
[0061] In certain embodiments, and as discussed below, vehicle 102 (e.g., as an embedded capability or via a wired or wireless connection to an individual's mobile cellular communication device) can include circuitry and processing resources capable of obtaining position-related measurements (e.g., for signals received from GPS or other satellite positioning system (SPS) satellites 1114), a cellular transceiver 1110, or a local transceiver 1115 and possibly calculating a position location or estimated position of vehicle 102 based on these position-related measurements. In some embodiments, position-related measurements obtained by vehicle 102 can be transmitted to a position server, such as an enhanced serving mobile location center (E-SMLC) or SUPL location platform (SLP) (e.g., which can include servers such as server 1140), after which the position server can estimate or determine an estimated position of vehicle 102 based on these measurements. In Figure 11 embodiments, position-related measurements obtained at vehicle 102 can include measurements of signals 1124 received from satellites belonging to an SPS or global navigation satellite system (GNSS) (such as GPS, GLONASS, Galileo, or Beidou), and / or can include measurements of signals (such as 1123 and / or 1125) received from ground transmitters fixed at known locations (e.g., such as cellular transceiver 1110).
[0062] Vehicle 102 (either as an embedded capability or via a wired or wireless connection to an individual's mobile cellular communication device) may use any one of several positioning methods such as, for example, GNSS, assisted GNSS (A-GNSS), advanced forward link trilateration (AFLT), observed time difference of arrival (OTDOA), or enhanced cell ID (E-CID), or a combination thereof, etc., to obtain a position estimate of vehicle 102 based on location-related measurements. In some of these techniques (e.g., A-GNSS, AFLT, and OTDOA), the pseudorange or time difference may be measured at vehicle 102 relative to three or more ground transmitters fixed at known locations or relative to four or more satellites with accurately known orbital data, or a combination thereof, based at least in part on a pilot, positioning reference signal (PRS), or other positioning-related signals transmitted by a transmitter or satellite and received at vehicle 102. Here, server 1140 may be capable of providing positioning assistance data to vehicle 102 (or to the individual's mobile cellular communication device within vehicle 102), including, for example, information about the signals to be measured (e.g., signal timing), the location and identity of the ground transmitters, and / or GNSS satellite signals, timing, and orbital information, to facilitate positioning techniques such as A-GNSS, AFLT, OTDOA, and E-CID. For example, server 1140 may include an almanac to indicate the location and identity of cellular transceivers and / or local transceivers in one or more specific areas (such as a particular venue), and may provide information describing the signals transmitted by a cellular base station or AP, such as transmit power and signal timing. In the case of E-CID, vehicle 102 may obtain measurements of the signal strength of the signals received from cellular transceiver 1110 and / or local transceiver 1115, and / or may obtain the round-trip signal propagation time (RTT) between vehicle 102 and cellular transceiver 1110 or local transceiver 1115. Vehicle 102 may use these measurements together with the assistance data received from server 1140 (e.g., ground almanac data or GNSS satellite data such as GNSS almanac and / or GNSS ephemeris information) to determine a position estimate of vehicle 102, or may transmit these measurements to server 1140 to perform the same determination. Calls from vehicle 102 may be routed based on the location of vehicle 102 via wireless communication link 1123 and communication link 1160.
[0063] In response to receiving signals from a GPS or from other satellite positioning system (SPS) satellites, or in response to other positioning methods (such as those described above), vehicle 102 may, for example, calculate or estimate its position. In a particular embodiment, the result of the position estimation process may be expressed using three variables, such as latitude, longitude, and altitude. However, in a particular embodiment, the estimated or calculated position may take any other form, such as expressing coordinates in Universal Transverse Mercator coordinates, or may be expressed using coordinates conforming to the World Geodetic System 84 (WGS 84), or may be expressed using any other coordinate system, and the claimed subject matter is not limited in this regard.
[0064] In response to vehicle 102 (either as an embedded capability or via an interface to an individual's mobile cellular communication device), an embedded sensor suite may be included, which may include, for example, inertial sensors and environmental sensors. The inertial sensors may include, for example, accelerometers (e.g., jointly responsive to the acceleration of vehicle 102 in the x, y, and z directions). Vehicle 102 may also include one or more gyroscopes or one or more magnetometers (e.g., to support one or more compass applications). The environmental sensors of vehicle 102 may include, for example, a temperature sensor, a barometric pressure sensor, an ambient light sensor, a camera imager, a microphone (to name just a few examples). The sensors of vehicle 102 may generate analog or digital signals, which may be stored using one or more memory locations to support one or more applications, such as, for example, an application that collects or obtains biometric identification attributes of an individual driver, for example.
[0065] Figure 12 is a diagram showing a computing environment according to Embodiment 1200. In Figure 12 an embodiment, a first device 1202, which may include an augmented reality device (e.g., augmented reality glasses), may, for example, receive a signal from a third device 1206, which may correspond to a content generator, such as Figure 6 content generator 205. In Figure 12 an embodiment, the first device 1202 may be capable of presenting a signal from the third device 1206 (e.g., the content generator) such that the driver and / or passengers of vehicle 102 may receive text and / or images related to or associated with real objects visible through the windshield (e.g., Figure 3 windshield 210) and / or the augmented reality glasses 145 (e.g.). A second device 1204 may potentially operate as an in-vehicle content formatter 135 (of Figure 1 ) that may generate context-sensitive content viewable by the driver and / or passengers of the vehicle. In Figure 12In this case, a computing device 1202 (“first device”) can communicate with a second device 1204, which may for example also include features of at least one computer processor coupled to at least one memory device and / or a server computing device. A processor (e.g., including one or more computer processors) 1220 and a memory 1222 (which may include a main memory 1225 and a secondary memory 1226) can communicate, for example, via a communication interface 1230. The term “computing device” or “computing resource” in this patent application refers to a system and / or device that includes the ability to process (e.g., perform calculations) and / or store digital content in the form of signals and / or states (such as electronic files, electronic documents, measurements, text, images, videos, audio, etc.), such as a computing device. Thus, in the context or environment of this patent application, a computing device can include hardware, software, firmware, or any combination thereof (except software itself). As Figure 12 The second device 1204 depicted in
[0066] In Figure 12 this case, the computing device 1202 can provide, for example, one or more sources of executable computer instructions in the form of a physical state and / or signal (e.g., stored in a memory state). The computing device 1202 can communicate with the computing device 1204 via a network connection (such as, for example, via a network 1208). As previously mentioned, although the connection is physical, it can be virtual and does not have to be tangible. Although Figure 12 the second device 1204 of
[0067] shows various tangible physical components, the claimed subject matter is not limited to computing devices having only these tangible components, because other specific implementations and / or embodiments can include, for example, alternative arrangements that function differently when achieving similar results, and these alternative arrangements can include additional tangible components or fewer tangible components. Instead, the examples are provided only for illustration. It is not intended to limit the scope of the claimed subject matter to the illustrative examples.
[0068] The memory 1222 may include one or more articles for storing programs of executable computer instructions. For example, the processor 1220 (which may include one or more computer processors) may obtain executable instructions from the memory and proceed to execute the obtained instructions. The memory 1222 may also include a memory controller for accessing the device-readable medium 1240, which may carry digital content and / or make digital content accessible, and the digital content may include, for example, code and / or instructions executable by the processor 1220 and / or some other device capable of executing, such as computer instructions, such as a controller, as an example. Under the guidance of the processor 1220 (e.g., one or more computer processors), a non-transitory memory, such as a memory unit storing physical states (e.g., memory states), including, for example, a program of executable computer instructions, may be executed by the processor 1220 and be capable of generating signals to be communicated via, for example, a network, as previously described. The generated signals may also be stored in the memory, as also previously mentioned.
[0069] The memory 1222 may store electronic files and / or electronic documents, such as those related to one or more users, and may also include a machine-readable medium, which may carry content and / or make content accessible, and the content includes, for example, code and / or instructions executable by the processor 1220 and / or some other device capable of executing, such as computer instructions, such as a controller, as an example. As previously mentioned, the term electronic file and / or the term electronic document are used throughout the document to refer to a set of stored memory states and / or a set of physical signals that are associated in such a way as to thereby form an electronic file and / or an electronic document. That is, no particular syntax, format, and / or method used with respect to a set of associated memory states and / or a set of associated physical signals is meant to be implicitly referenced. It should also be noted that the association of memory states may be, for example, in a logical sense, and not necessarily in a tangible, physical sense. Thus, in one embodiment, although the signal and / or state components of an electronic file and / or an electronic document will be logically associated, their storage may, for example, reside in one or more different locations in a tangible physical memory.
[0070] Algorithmic descriptions and / or symbolic representations are examples of techniques used by those of ordinary skill in the signal processing and / or related arts to convey the substance of their work to other artisans in the field. In the context or environment of this patent application, an algorithm is generally considered to be a self-consistent sequence of operations and / or something like signal processing that results in a desired outcome. In the context or environment of this patent application, operations and / or processing involve the physical manipulation of physical quantities. Typically, although not necessarily, such quantities may take the form of electrical and / or magnetic signals and / or states that can be stored, transmitted, combined, compared, processed, and / or otherwise manipulated, such as electronic signals and / or states that are components of various forms of digital content (such as signal measurements, text, images, video, audio, etc.).
[0071] Processor 1220 may include one or more circuits, such as digital circuits, to perform at least a portion of a computing program and / or process. By way of example and not limitation, processor 1220 may include one or more processors, such as a controller, microprocessor, microcontroller, application specific integrated circuit, digital signal processor, programmable logic device, field programmable gate array, etc., or any combination thereof. In various specific implementations and / or embodiments, processor 1220 may generally perform signal processing substantially in accordance with the acquired executable computer instructions, such as to manipulate signals and / or states, construct signals and / or states, etc., where the signals and / or states generated in this manner are communicated and / or stored in, for example, a memory.
[0072] Figure 12 Device 1204 is also shown as including, for example, components 1232 that may operate with input / output devices and a communication bus 1215 such that signals and / or states may be appropriately communicated between devices such as between device 1204 and an input device and / or between device 1204 and an output device. A user may use an input device, such as a computer mouse, stylus, trackball, keyboard, and / or any other similar device capable of receiving user actions and / or movements as input signals. Similarly, for a device with voice-to-text capabilities, the user may speak to generate an input signal. Similarly, the user may use an output device, such as a display, printer, etc., and / or any other device capable of providing signals and / or generating stimuli (such as visual stimuli, audio stimuli, and / or other similar stimuli) to the user.
[0073] Unless otherwise indicated, in this patent application, the term "or" (when used in connection with a list such as A, B, or C) is intended to mean "A, B, and C" (used here in an inclusive sense), as well as "A, B, or C" (used here in an exclusive sense). Under this understanding, "and" is used in an inclusive sense and is intended to mean A, B, and C; while "and / or" may be used sparingly to clearly indicate that all of the foregoing meanings are intended, although such usage is not required. Additionally, the terms "one or more" and / or similar terms are used to describe any feature, structure, characteristic, etc. in the singular, and "and / or" is also used to describe multiple features, structures, characteristics, etc. and / or some other combination of features, structures, characteristics. Similarly, the terms "based on" and / or similar terms are understood to not necessarily be intended to convey an exhaustive list of factors, but rather to allow for the presence of additional factors that may not be explicitly described.
[0074] In the foregoing description, various aspects of the claimed subject matter have been described. For purposes of explanation, details such as amounts, systems, and / or configurations have been set forth as examples. In other instances, well-known features have been omitted and / or simplified so as not to obscure the claimed subject matter. Although certain features have been illustrated and / or described herein, many modifications, substitutions, changes, and / or equivalents will now occur to those skilled in the art. Accordingly, it is to be understood that the appended claims are intended to cover all modifications and / or changes that fall within the scope of the claimed subject matter.
Claims
1. An augmented reality controller for transmitting content to an augmented reality device communicating with a vehicle, the augmented reality controller comprising: at least one memory device storing computer program code; one or more processors communicatively coupled to the at least one memory device and configured to, when executing the computer program code: estimate the position of the augmented reality device relative to one or more known regions of a grid; determine the orientation of the field of view of the augmented reality device; and transmit context-sensitive content to the augmented reality device via the vehicle for display on the augmented reality device, the context-sensitive content being associated with one or more real-world objects visible within the field of view of the augmented reality device and generated at least in part based on the one or more known regions of the grid.
2. The augmented reality controller according to claim 1, wherein the one or more processors communicatively coupled to the at least one memory device are further configured to: access a cloud-based data storage device to determine the context-sensitive content at least in part based on one or more content preferences of an individual currently in the vehicle or meeting predefined proximity conditions relative to the vehicle.
3. The augmented reality controller according to claim 2, wherein the one or more processors communicatively coupled to the at least one memory device are further configured to: receive a user's selection of a travel type context from a plurality of selectable travel type contexts, where the travel type context indicates a travel mode in which the individual is participating or indicates whether the individual has a vehicle operator role or a vehicle passenger role, wherein the data storage device is accessed at least in part based on the user's selection of the travel type context such that the context-sensitive content is based on the travel type context.
4. The augmented reality controller according to claim 3, wherein, in response to the user's selection of the travel type context, the one or more processors communicatively coupled to the at least one memory device are further configured to: transmit the context-sensitive content to the augmented reality device via the vehicle at least in part based on the positioning of the one or more real-world objects along a driving travel path of the vehicle.
5. The augmented reality controller according to claim 3, wherein, in response to selecting the travel type context, the one or more processors communicatively coupled to the at least one memory device are further configured to: transmit the context-sensitive content to the augmented reality device via the vehicle at least in part based on the positioning of the one or more real-world objects along a walking travel path.
6. The augmented reality controller according to claim 1, wherein the one or more processors communicatively coupled to the at least one memory device are configured to perform the estimation of the position of the augmented reality device by: Receive a selection of at least one of the one or more known regions of the grid from among a plurality of uniformly sized regions within a predefined proximity of an estimated position of the vehicle.
7. The augmented reality controller according to claim 6, wherein the plurality of uniformly sized regions correspond to regions of approximately 3.0 m by approximately 3.0 m.
8. The augmented reality controller according to claim 1, wherein the one or more processors communicatively coupled to the at least one memory device are further configured to: Determine whether one or more real-world objects visible within the field of view of the augmented reality device meet a predefined proximity condition in which the one or more real-world objects are considered to be relatively close to an individual co-located with the augmented reality device, or whether the one or more real-world objects meet a predefined remote condition in which the one or more real-world objects are considered to be relatively far from the individual, wherein the context-sensitive content is generated based on whether the one or more real-world objects meet the predefined proximity condition or whether the one or more real-world objects meet the predefined remote condition.
9. The augmented reality controller according to claim 1, wherein the one or more processors communicatively coupled to the at least one memory device are further configured to: Determine whether one or more real-world objects visible within the field of view of the augmented reality device are located within a peripheral portion of the field of view of the augmented reality device or within a central portion of the field of view of the augmented reality device, wherein the central portion of the field of view is the portion within a first predefined angle relative to a central axis of the field of view, and wherein the peripheral portion is the other portion outside the central portion of the field of view.
10. The augmented reality controller according to claim 1, wherein the one or more processors communicatively coupled to the at least one memory device are further configured to: Receive from a graphical user interface of the vehicle a user selection to display the context-sensitive content using alphanumeric characters, graphical icons, or a combination thereof.
11. A method for providing content to an augmented reality device communicatively coupled to a vehicle, the method being executed by one or more processors and comprising: Estimate a position of the augmented reality device relative to one or more known regions of a grid; Determine an orientation of a field of view of the augmented reality device; and Generate and / or transmit via the vehicle context-sensitive content associated with one or more real-world objects visible within the field of view of the augmented reality device based on the one or more known regions of the grid.
12. The method according to claim 11, the method further comprising: Access a data storage device to determine the context-sensitive content based at least in part on one or more content preferences of an individual currently in the vehicle or meeting a predefined proximity condition relative to the vehicle.
13. The method according to claim 12, the method further comprising: Receive a selection by a user of a travel type context of the individual from among a plurality of travel type contexts, where the travel type context indicates a travel mode in which the individual is participating or indicates whether the individual has a vehicle operator role or a vehicle passenger role, and where the content is generated based on the travel type context.
14. The method according to claim 13, the method further comprises: In response to selecting the travel type context, emitting the context-sensitive content associated with the one or more real objects, at least in part based on a positioning of the one or more real objects along a driving travel path of the vehicle.
15. The method according to claim 11, the method further comprises: Receive a selection of the one or more known regions of the grid from among a plurality of uniformly sized regions within a predefined proximity of the location of the augmented reality device and / or the vehicle.
16. The method according to claim 15, wherein the plurality of uniformly sized regions correspond to regions of approximately 3.0 m by approximately 3.0 m.
17. A non-transitory computer-readable medium comprising program instructions for causing one or more processors of an augmented reality controller to perform at least the following operations: Estimate a position of the augmented reality device relative to one or more known regions of a grid; Determine an orientation of a field of view of the augmented reality device; and Emit context-sensitive content via the vehicle to the augmented reality device, the context-sensitive content being associated with one or more real objects visible within the field of view of the augmented reality device and being generated at least in part based on the one or more known regions of the grid.
18. The non-transitory computer-readable medium according to claim 17, wherein the program instructions further cause the one or more processors to: Access a cloud-based data storage device to determine the context-sensitive content at least in part based on one or more content preferences of an individual currently in the vehicle or meeting a predefined proximity condition relative to the vehicle.
19. The non-transitory computer-readable medium according to claim 18, wherein the program instructions further cause the one or more processors to: Receive a selection by a user of a travel type context from among a plurality of selectable travel type contexts, where the travel type context indicates a travel mode in which the individual is participating or indicates whether the individual has a vehicle operator role or a vehicle passenger role, and where the data storage device is accessed at least in part based on the user's selection of the travel type context such that the context-sensitive content is based on the travel type context.
20. The non-transitory computer-readable medium according to claim 17, wherein the program instructions further cause the one or more processors to: Determine whether the one or more real-world objects visible within the field of view meet a predefined proximity condition in which the one or more real-world objects are considered to be relatively close to an individual co-located with the augmented reality device, or whether the one or more real-world objects meet a predefined remote condition in which the one or more real-world objects are considered to be relatively far from the individual, wherein the context-sensitive content is generated based on whether the one or more real-world objects meet the predefined proximity condition or whether the one or more real-world objects meet the predefined remote condition.