Position determination for mobile computing devices

Abstract

Methods, systems, devices, and tangible non-transitory computer-readable media for determining position are provided. The disclosed technology can include capturing an image of an environment visible in a field of view of a mobile computing device. Location data associated with a location of the mobile computing device can be received. Image data including information associated with the image can be generated. Based on the image data and one or more machine learning models, characteristics of one or more objects in the environment can be determined. Based on the location data and the characteristics of the objects, a corresponding position of the objects relative to the location of the mobile computing device can be determined. In addition, position data including a geographic position of the mobile computing device can be generated based on the corresponding (geographic) position of the objects.

Description

Technical Field

[0001] The present disclosure relates generally to navigation, and in particular to systems, devices, and methods for determining a position including a position of a computing device. Background Art

[0002] Operations associated with the state of a geographic area can be performed on a variety of computing devices. These operations can include processing data associated with the geographic area for later access and use by a user or computing system. In addition, the operations can include exchanging data with a remote computing system. However, the types of operations performed and the manner in which the operations are performed may change over time, as may the underlying hardware that performs the operations. Therefore, there are different ways to affect computing resources associated with the state of a geographic area. Summary of the invention

[0003] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0004] One example aspect of the present disclosure relates to a computer-implemented method for navigation and position determination. The computer-implemented method can include: capturing, by a mobile computing device including one or more processors, one or more images of an environment visible in a field of view of the mobile computing device. In addition, the computer-implemented method can include: receiving, by the mobile computing device, location data associated with a location of the mobile computing device. The computer-implemented method can include: generating, by the mobile computing device, image data including information associated with the one or more images. The computer-implemented method can also include: determining, by the mobile computing device, one or more object classifications of one or more objects in the environment based at least in part on the image data and one or more machine learning models. The computer-implemented method can include. Determining, by the mobile computing device, one or more corresponding positions of one or more objects relative to the location of the mobile computing device based at least in part on the location data and one or more object classifications of the one or more objects. In addition, the computer-implemented method can include: generating, by the mobile computing device, position data including a geographic position of the mobile computing device based at least in part on one or more corresponding positions of the one or more objects.

[0005] Another example aspect of the present disclosure relates to one or more tangible non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform operations. The operations can include capturing one or more images of an environment visible in a field of view of a mobile computing device. In addition, the operations can include receiving location data associated with a location of the mobile computing device. The operations can include generating image data including information associated with the one or more images. The operations can also include determining one or more object classifications of one or more objects in the environment based at least in part on the image data and one or more machine learning models. The operations can include determining one or more corresponding positions of one or more objects relative to the location of the mobile computing device based at least in part on the location data and one or more object classifications of the one or more objects. In addition, the operations can include generating position data including a geographic position of the mobile computing device based at least in part on one or more corresponding positions of the one or more objects.

[0006] Another example aspect of the present disclosure relates to a computing system that can include: one or more processors; and one or more tangible non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations can include capturing one or more images of an environment visible in a field of view of a mobile computing device. In addition, the operations can include receiving location data associated with a location of the mobile computing device. The operations can include generating image data including information associated with the one or more images. The operations can also include determining one or more object classifications of one or more objects in the environment based at least in part on the image data and one or more machine learning models. The operations can include determining one or more corresponding positions of one or more objects relative to the location of the mobile computing device based at least in part on the location data and one or more object classifications of the one or more objects. In addition, the operations can include generating position data including a geographic position of the mobile computing device based at least in part on the one or more corresponding positions of the one or more objects.

[0007] Other example aspects of the present disclosure relate to other methods, systems, devices, apparatuses or tangible non-transitory computer-readable media for navigation and position determination. Moreover, it will be understood that any feature described with respect to one example aspect can also be implemented as part of another example aspect.

[0008] These and other features, aspects and advantages of various embodiments will become better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] A detailed discussion of the embodiments is set forth in the specification, which is relevant to those skilled in the art, with reference to the accompanying drawings, in which:

[0010] Figure 1 A diagram depicting an example system according to an example embodiment of the present disclosure;

[0011] Figure 2 A diagram depicting an example device according to an example embodiment of the present disclosure;

[0012] Figure 3 Describing an example of position determination in an environment according to an example embodiment of the present disclosure;

[0013] Figure 4 Describing an example of position determination in an environment according to an example embodiment of the present disclosure;

[0014] Figure 5 Depicting an example of a device that performs position determination according to an example embodiment of the present disclosure;

[0015] Figure 6 depicts a flow chart of orientation determination according to an example embodiment of the present disclosure;

[0016] Figure 7 depicts a flow chart of orientation determination according to an example embodiment of the present disclosure;

[0017] Figure 8 depicts a flow chart of orientation determination according to an example embodiment of the present disclosure;

[0018] Fig. 9 depicts a flow chart of position determination according to an example embodiment of the present disclosure; and

[0019] Fig.10 A flow chart of position determination is depicted according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] Reference will now be made in detail to the embodiments, one or more examples of which are illustrated in the accompanying drawings. Each example is provided as an explanation of the embodiments rather than a limitation of the present disclosure. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments without departing from the scope or spirit of the present disclosure. For example, a feature illustrated or described as a part of one embodiment can be used in another embodiment to produce other embodiments. Therefore, aspects of the present disclosure are intended to cover such modifications and variations.

[0021] An example aspect of the present disclosure relates to a device that can be used to determine the position of one or more objects in an environment surrounding the device. Additionally, the disclosed techniques can be used to determine the position of the device by using a combination of location data associated with an approximate location of the device, image data associated with the appearance of the environment surrounding the device, and at least one machine learning model that can be trained to determine features of objects in the environment surrounding the device.

[0022] Specifically, the disclosed technology can determine the approximate location (rough geographic location) of a device using signals from navigation satellites (e.g., global positioning system ("GPS") satellites). By accessing additional data associated with characteristics of the location (e.g., the street on which the device is located), the approximate location (or location data) of the device can be used to determine whether the street is a one-way street or a two-way street. In addition, the device can capture an image of the street and / or the environment. The image data of the captured image can be provided as an input to a machine learning model, which can use the captured image to generate an output including a direction facing an object (e.g., a parked vehicle) in the image. Using information including whether the street is one-way or two-way and the orientation of the objects in the image, the device can determine the orientation of the device with respect to at least one object. The geographic orientation of the device can then be determined based on the relative orientation of the object with respect to the device. In this manner, the disclosed technology can provide users with a reasonably accurate estimate of their orientation, allowing for more efficient navigation through the environment. This approach can have particular benefits in urban areas where GPS positioning systems are less accurate due to the interference effects of buildings but where there are many objects in the environment. Additionally, the disclosed technology can provide additional benefits by providing improved location services, including a reduction in the amount of time and energy required to travel from one place to another.

[0023] The disclosed technology can be implemented by a computing device (e.g., a mobile computing device) and / or a computing system (e.g., a position computing system) configured to access data, perform operations on the data (e.g., determine the position of the device with respect to nearby objects), and generate an output including the position of the computing device. In addition, the computing system can be included in the mobile computing device and / or as part of a system that includes a navigation server that: receives and / or sends location data (e.g., the current position of the mobile device) and / or image data (e.g., information and / or data associated with one or more images of the environment surrounding the mobile device) from a client device (e.g., a mobile device that can include a smart phone); performs operations based on the received data; and generates and / or sends output including the position of the device back to the client device, which can be configured to use the determined position to perform additional operations.

[0024] As another example, the disclosed technology can use one or more cameras of a mobile computing device to capture one or more images of an environment visible in the field of view of the mobile computing device. The mobile computing device can then receive location data associated with the location of the mobile computing device (e.g., latitude, longitude, and / or altitude that can be associated with a street address). In addition, image data including information associated with one or more images can be generated. The one or more images can, for example, include depictions of objects in the environment and can be formatted to be used as input to a machine learning model. One or more machine learning models (e.g., machine learning models trained to detect, identify, and / or recognize visible features in an image) can then receive inputs including image data and use the inputs to generate outputs including one or more object classifications of one or more objects in the environment depicted in the image data. In addition, based at least in part on the location data and / or one or more object classifications of one or more objects, the mobile computing device can determine one or more corresponding positions of one or more objects relative to the location of the mobile computing device. Position data including the geographic position of the mobile computing device can then be generated based at least in part on one or more corresponding (geographic) positions of the one or more objects. The geographic position can include, for example, a compass-determined position and / or associated with a compass point (e.g., cardinal directions (north, south, east, west), secondary cardinal directions (northeast, southeast, southeast, southwest), and / or secondary secondary cardinal directions). Additionally, the position data can be included as part of other data that can be used by other devices and / or systems, including vehicle systems.

[0025] Thus, the disclosed technology can improve the determination of position and location in (urban) environments and thus improve navigation efficiency and safety by providing users with a more accurate indication of their location. Furthermore, the disclosed technology can assist users in more efficiently performing technical tasks using the location of a device to facilitate navigation from one location to another by means of a continuous and / or guided human-machine interaction process.

[0026] A computing device (e.g., a mobile computing device) can capture one or more images of an environment visible in a field of view of the mobile computing device. For example, the mobile computing device can include one or more cameras with one or more corresponding fields of view. Each of the one or more cameras can be used to capture one or more images (e.g., color images) of the environment in which the camera is present. Additionally, in some embodiments, the mobile computing device can be a handheld device (e.g., a smart phone) that is pointed in a direction to be determined and / or is aligned in that direction.

[0027] In some embodiments, the mobile computing device can include one or more image capture devices. For example, the mobile computing device can include one or more cameras that can be used to capture one or more images of the environment. Image capture devices can include various types of cameras with different focal lengths, different sensor types (e.g., sensors configured to capture visible light and / or infrared light), and / or different apertures. In addition, the mobile computing device can include other types of sensors that can be used to determine the location and / or orientation of the mobile computing device, including radar, sonar, and / or other passive and / or active sensor systems.

[0028] The mobile computing device can receive location data associated with the location of the mobile computing device. In some embodiments, the location data can be based at least in part on or include output from a satellite navigation system and / or output from one or more wireless communication network devices. For example, the mobile computing device can receive location data based at least in part on one or more signals from GPS and / or GLONASS satellites. The location data can include information associated with the latitude, longitude, and / or altitude of the mobile computing device. In addition, the location data can be based at least in part on data and / or information from a computing system that can provide additional information associated with the location of the mobile computing device. The information included in the location data provided to the mobile computing system can include an address, neighborhood, city, town, and / or municipality associated with the location of the mobile computing device. Additionally, the location data can include information associated with a route of travel (e.g., a street, road, or other path) that can be used to determine a reference point from which the orientation of one or more objects along the route of travel can be determined. In some embodiments, the mobile computing device can store one or more portions of the location data locally.

[0029] The mobile computing device can generate image data including information associated with one or more images. For example, the mobile computing device can perform one or more operations to generate image data based at least in part on output from one or more sensors associated with one or more image capture devices that captured the one or more images. In some embodiments, the image data can be encoded in various ways, including in a raster (e.g., bitmap), voxel, or vector image format. In addition, the image data can be based at least in part on a video including one or more images. For example, the image data can be extracted from a plurality of sequential images of a video. In addition, the image data can be generated in a format that can be provided as an input to a machine learning model.

[0030] The mobile computing device can determine one or more object classifications of one or more objects in the environment based at least in part on the image data and one or more machine learning models. For example, the mobile computing device can include one or more machine learning models that can be configured and / or trained to: receive an input including image data; perform one or more operations on the input including extracting one or more features from the image data; and generate an output including one or more object classifications and / or features of the one or more objects. For example, one or more features of a portion of an image depicted in the image data can be classified as a portion of a motor vehicle object including a windshield, a door, a wheel, or a license plate.

[0031] In some embodiments, one or more machine learning models can be configured and / or trained based at least in part on training data, the training data including one or more images of one or more vehicles at one or more distances and one or more angles with respect to an image capture device that captured one or more images of the one or more vehicles. For example, the training data used to configure and / or train one or more machine learning models can include one or more images of different vehicles (e.g., vehicles of different types, brands, and / or models) captured from three hundred and sixty degrees around the vehicle; from different heights including viewing the front, above, or below the vehicle; at different angles; under different lighting conditions (e.g., midday sunlight, dawn light, dusk light, moonlight, artificial light (e.g., street lights, light in an indoor parking garage, and / or light in a tunnel); and / or in situations where portions of the vehicle are obscured or blocked (e.g., a vehicle with features that are muddied or blocked by another vehicle, a pedestrian, or a tree branch). A more robust machine learning model can thus be provided.

[0032] In some embodiments, the location of the mobile computing device can be used to determine one or more machine learning models to be used. In addition, the location of the mobile computing device can be used to determine one or more machine learning models including at least one machine learning model that is configured and / or trained based at least in part on training data associated with a geographic area including the location of the mobile computing device. For example, each of the one or more machine learning models can be trained based at least in part on a corresponding training data set associated with a specific geographic area (e.g., a different geographic area where signage, language, traffic rules, and / or road signs are different). In addition, the mobile computing device can determine and / or select a machine learning model associated with the area including the location of the mobile computing device. In some embodiments, one or more machine learning models can be trained based at least in part on any combination of multiple training data sets associated with multiple different geographic areas. For example, a single machine learning model can be trained using training data sets associated with three different geographic areas in a single country. This approach can facilitate improved device position determination by taking into account regional differences that can affect the determination of the position of an object in the environment of the device.

[0033] In some embodiments, one or more machine learning models can be configured and / or trained based at least in part on training data associated with one or more images of one or more different geographic regions. In addition, each of the one or more geographic regions can be associated with a corresponding plurality of traffic regulations. For example, the plurality of traffic regulations can include one or more regulations associated with: the side of a street that a vehicle can be parked on during certain hours; whether one-way travel, two-way travel, or some other traffic flow pattern is allowed on a route of travel; and / or the permissible direction of travel along a portion of a route of travel. This approach can facilitate improved device position determination by taking into account geographic differences that can affect the determination of the position of an object in the device's environment.

[0034] In some embodiments, the one or more objects can include one or more vehicles. For example, the one or more objects depicted in the image data can include one or more motor vehicles, including cars, buses, trains, trams, trolleybuses and / or streetcars.

[0035] In some embodiments, one or more machine learning models can be configured and / or trained to detect or recognize one or more visible features associated with image data. For example, the one or more machine learning models can: receive an input including image data; perform one or more operations associated with detection, recognition, and / or identification of one or more visible features of an image depicted in the image data; and generate classification information associated with a classification of the one or more visible features of an object. For example, the output can include a determination that the one or more visible features include rear lights (e.g., tail lights) of a vehicle, which can then be used to determine the location of the vehicle relative to a device within an environment, and which can then be used to determine the orientation of the mobile device that provided the image data.

[0036] In some embodiments, one or more machine learning models can be configured and / or trained based at least in part on training data associated with an image corpus, which can include multiple two-dimensional images of one or more environments captured from a land-based image capture device. For example, the image corpus can include multiple labeled images of various streets. In addition, the labeled images can include semantic information associated with the location, name, and address of the street. In addition, the labeled images can include information associated with one or more objects depicted in the labeled images, including the geographic location (e.g., latitude, longitude, and / or altitude) of the one or more objects and / or the orientation of each of the one or more objects with respect to the image capture device that captured the labeled image.

[0037] Training one or more machine learning models can include providing multiple images of objects in various environments as input to the one or more machine learning models. The one or more machine learning models can then perform one or more operations on the input, including: extracting one or more features of the input and generating an output, the output including a classification of the object and / or object features depicted in the image. The output of the one or more machine learning models can then be evaluated based at least in part on one or more comparisons of the classified objects and corresponding ground-truth labels associated with each object. In some embodiments, the output of the one or more machine learning models can include an estimated distance of one or more objects from an image capture device that captures the multiple images, an estimate of the relative orientation of the object with respect to the image capture device that captures the multiple images, and / or an estimate of the size of the object.

[0038] Based at least in part on one or more comparisons of the classified objects and the corresponding true annotations associated with each object, the associated loss can be determined based at least in part on an evaluation of a loss function associated with the accuracy of the output generated by the one or more machine learning models. In addition, the one or more machine learning models can include one or more parameters that can be associated with analyzing, processing, determining, generating and / or extracting one or more features of the input (e.g., one or more features associated with the image data). Then, one or more parameters of the one or more machine learning models can be adjusted based at least in part on the contribution of each of the one or more parameters to minimizing the loss. In this way, the one or more parameters of the one or more machine learning models can be adjusted through multiple iterations so that the loss is reduced and the accuracy of the object classification of the one or more machine learning models is improved.

[0039] By way of example, one or more parameters of one or more machine learning models that do not change the loss (or do not change the loss by more than a threshold amount) can be left intact; one or more parameters that reduce the loss (e.g., are adjusted to increase their contribution to the loss) can be weighted more heavily; and / or one or more parameters that increase the loss can have their weighting reduced (e.g., the value associated with the corresponding parameter is adjusted to reduce the parameter's contribution to the loss).

[0040] In some embodiments, one or more machine learning models can include various types of machine learning models (e.g., convolutional neural networks, recurrent neural networks, and / or recursive neural networks). In addition, one or more machine learning models can include different numbers of layers, such as a single layer or multiple layers.

[0041] The mobile computing device can determine one or more corresponding orientations of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and / or one or more object classifications of the one or more objects. For example, the location data can indicate that the mobile computing device is on a one-way road at an orientation of ten degrees east of geographic north. Additionally, the mobile computing device can determine that the one or more object classifications of objects in the field of view of the mobile computing device are associated with a rear bumper of a motor vehicle that is perpendicular to the one-way road, and that the orientation at which the bumper is attached to the motor vehicle is approximately ten degrees east of geographic north.

[0042] In some embodiments, determining one or more corresponding orientations of one or more objects relative to the position of the mobile computing device can include: determining one or more sides of the one or more objects based at least in part on one or more object classifications. The one or more sides of the one or more objects can include a front side, a back side, a top side, a bottom side, a left side, and / or a right side. In addition, one or more machine learning models can be configured and / or trained to determine one or more sides of an object facing an image capture device that captures one or more images of the object.

[0043] Additionally, the mobile computing device can determine one or more corresponding positions of one or more sides of the one or more objects relative to the mobile computing device. For example, the mobile computing device can determine that a first object facing the mobile computing device is a vehicle and that a rear side of the vehicle is directly in front of the mobile computing device. Additionally, the mobile computing device can determine that a second object facing the mobile computing device is a second vehicle and that a side of the second vehicle is in front of the mobile computing device, perpendicular to the first vehicle and arranged from right to left across the field of view of the mobile computing device. Additionally, the mobile computing device can determine that the mobile computing device is on the right side of the street. Based on the position data indicating that the mobile computing device is on a two-way street and the sides of the objects depicted in the image data (e.g., the first vehicle, the second vehicle, and the street), the mobile computing device can determine the position of the mobile computing device relative to the one or more objects.

[0044] In some embodiments, one or more sides of one or more objects can include one or more sides of one or more vehicles (e.g., one or more motor vehicles). For example, one or more sides of one or more objects can include the front side of a vehicle, the rear side or tail side of a motor vehicle, the left side of a motor vehicle, or the right side of a vehicle. In addition, one or more sides of one or more objects can include the left side of a route of travel or the right side of a route of travel.

[0045] In some embodiments, one or more object classes can be associated with one or more vehicles. Additionally, the one or more object classes can include one or more headlight classes, one or more taillight classes, one or more front windshield classes, one or more rear windshield classes, one or more front grille classes, one or more wheel classes, one or more side window classes, one or more door classes, and / or one or more bumper classes.

[0046] As part of determining one or more corresponding positions of one or more objects, when one or more objects include one or more lights associated with one or more objects (e.g., headlights and / or taillights of a vehicle in an illuminated state or a dark state), the mobile computing device can determine based at least in part on one or more object classifications. In addition, the mobile computing device can determine one or more corresponding positions of the one or more lights relative to the location of the mobile computing device. For example, one or more machine learning models can be configured and / or trained to determine, recognize and / or identify one or more features associated with one or more lights associated with one or more vehicles, including various shapes, colors, and / or light intensity levels of one or more lights. Using lights in this manner can, for example, be combined with location data to provide an indication of the road the device is pointed to and / or which side of the street the device is located on based on which road the traffic flow is on. The determination of the geographic position of the device within an environment, particularly within an urban environment, can therefore be improved. This can be particularly beneficial at night when other visible signals indicating position may be difficult to identify.

[0047] In some embodiments, determining one or more respective positions of one or more objects relative to the location of the mobile computing device can include determining a path of travel associated with the environment based at least in part on one or more object classifications of the one or more objects. For example, one or more machine learning models can be configured and / or trained to determine, recognize, and / or identify one or more object classifications associated with the path of travel, including road signs, curbs, crosswalks, traffic lights, and / or various features of the path of travel itself (e.g., shapes, colors, and / or textures associated with asphalt, concrete, and / or other road surfaces).

[0048] Additionally, the mobile computing device can determine the position of the path of travel relative to the mobile computing device. For example, the mobile computing device can use geometric features and visible features associated with the path of travel (e.g., lines associated with edges of a road) as reference points with which to determine the position of the path of travel relative to the mobile device.

[0049] In some embodiments, the one or more object classifications associated with the one or more objects can include one or more indications associated with a direction of travel along a path of travel. For example, the one or more indications can include signs (e.g., a one-way street sign) and / or surface markings (e.g., arrows or the words "one-way" street or "two-way" street painted on the road surface).

[0050] In some embodiments, determining one or more corresponding positions of the one or more objects relative to the location of the mobile computing device can include determining one or more directions of travel of the one or more objects based at least in part on one or more object classifications of the one or more objects. The one or more directions of travel can be based at least in part on one or more movements of the one or more objects relative to the location of the mobile device. For example, one or more machine learning models can be configured and / or trained to determine a front side of a vehicle and, upon detecting an approaching vehicle, determine that the vehicle is traveling forward toward the mobile computing device.

[0051] On a one-way street, in some embodiments, the mobile computing device can determine the orientation of the mobile computing device based at least in part on the direction of travel of the vehicle (e.g., if the one-way street faces due south and the front of the vehicle is heading towards the mobile computing device, the mobile computing device orientation is due north).

[0052] On a two-way street, the mobile computing device can first determine the side of the street the mobile computing device is on. Assuming traffic rules require right-hand driving, if the mobile computing device is on the left side of the street and a vehicle is traveling north on the opposite side of the street, the rear of the vehicle is visible to the mobile computing device, then the heading of the mobile computing device is also north.

[0053] In some embodiments, determining one or more respective positions of the one or more objects relative to the location of the mobile computing device can include determining one or more location identifiers associated with the environment based at least in part on one or more object classifications of the one or more objects. As an example, the one or more location identifiers can include one or more street numbers indicating a number of a street, including whether the street number is odd or even.

[0054] The mobile computing device can then determine one or more corresponding positions of the one or more location identifiers relative to the mobile computing device. For example, the mobile computing device can use the one or more location identifiers to determine the street on which the mobile computing device is located, and can use the street number and / or whether the street number is odd or even to determine which side of the street the mobile computing device is on. Additionally, the mobile computing device can use the side of the street on which the mobile computing device is located to determine in which direction the position of the mobile computing device is based on whether the one or more street numbers are increasing or decreasing in value.

[0055] In some embodiments, the position of the mobile computing device can be based at least in part on one or more locations associated with the one or more location identifiers, respectively. For example, the one or more location identifiers can include a road sign located at the intersection of two streets. The location of the intersection within the field of view of the mobile computing device can be used to determine the approximate position of the mobile computing device based at least in part on the location of the one or more location identifiers. In addition, the position of the mobile computing device can be determined based at least in part on a spatial relationship (e.g., distance) between the one or more location identifiers and the mobile computing device.

[0056] In some embodiments, the one or more location identifiers can include one or more street numbers, one or more street names, and / or one or more signs associated with a geographic location.

[0057] In some embodiments, one or more location identifiers (e.g., road signs with address numbers) can be used to determine which side of a path of travel (e.g., a street) a mobile computing device is located on. For example, one or more machine learning models can be configured and trained to perform text recognition on numbers displayed on road signs. The side of the street that the mobile computing device is located on can be determined based at least in part on whether the recognized number is an odd number or an even number (e.g., in some geographic areas, odd numbers are always on one side of a path of travel and even numbers are always on the other side of the path of travel).

[0058] In some embodiments, determining one or more respective positions of the one or more objects relative to the location of the mobile computing device can include determining an amount of daylight in one or more portions of the environment (e.g., one or more different portions of the environment) based at least in part on the one or more features of the one or more objects. For example, the mobile computing device can analyze the image data to determine the amount of daylight in different portions of the image based at least in part on the brightness and / or intensity of each portion (e.g., pixel) of the image.

[0059] Additionally, the mobile computing device can determine one or more corresponding positions of one or more objects relative to the mobile computing device based at least in part on the amount of sunlight in one or more portions of the environment. For example, using an image in which objects on one side of a street are brightly illuminated by sunlight and the other side is less brightly illuminated by sunlight, the position of the sun can be estimated. The position of the sun can then be used as part of determining the position of the mobile computing device with respect to one or more objects in the environment.

[0060] In some embodiments, the orientation of the mobile computing device can be based at least in part on the moment when the amount of daylight in one or more parts of the environment is detected. For example, the image data can indicate the position of the sun in the sky, which can be used to partially determine the orientation of the mobile computing device. As another example, the amount of daylight detected on various surfaces (e.g., a ground surface or a nearby wall of a building) can be used to estimate the relative position of the sun, which in turn can be used to determine the orientation of the surface relative to the mobile computing device. In addition, the intensity of daylight projected on different surfaces can be used to estimate the relative position of the sun and thus determine the orientation of the mobile computing device. In some embodiments, the actual position of the sun can be used to determine one or more corresponding orientations of one or more objects.

[0061] Additionally, the orientation of the mobile computing device can be based at least in part on the direction of one or more shadows in the environment. For example, the direction of a shadow cast by a stationary object can change depending on the time of day, such that a shadow on one side of a stationary object at dawn will be on the other side of the object at dusk. Additionally, the direction of the one or more shadows can be used to estimate the relative position of the sun, which in turn can be used to determine the orientation of the mobile computing device.

[0062] In some embodiments, the mobile computing device can use the position of the moon and / or other celestial bodies as part of one or more corresponding positions of one or more objects relative to the position of the mobile computing device.

[0063] In some embodiments, determining one or more corresponding positions of the one or more objects relative to the position of the mobile computing device can include determining one or more directions of travel of the one or more objects along the path of travel. Determining the one or more directions of travel along the path of travel can be based at least in part on one or more positions of one or more forward portions of the one or more objects relative to the path of travel. For example, one or more machine learning models can determine that the image data includes a depiction of the front of a vehicle, and based on the position of the vehicle along the path of travel (e.g., a one-way street), the position of the vehicle can be determined. The position of the vehicle relative to the mobile computing device can then be used to determine the position of the mobile computing device.

[0064] The mobile computing device can generate and / or determine orientation data. The orientation data can include a geographic orientation of the mobile computing device. The geographic orientation of the mobile computing device can be based, at least in part, on one or more corresponding (geographic) orientations of one or more objects (e.g., relative to the orientation of the device). For example, the mobile computing device can determine that the rear orientation of the object is north (e.g., geographic north or magnetic north). Additionally, the mobile computing device can determine that one or more object classifications of objects in the field of view of the mobile computing device are associated with a rear bumper of a motor vehicle perpendicular to the one-way road, the orientation of the motor vehicle to which the bumper is attached being approximately ten degrees east of the north orientation of the one-way road.

[0065] As described herein, an orientation can include the position or orientation of one or more objects with respect to some reference point, which can include the geographic North Pole, the magnetic North Pole, the geographic South Pole, the magnetic South Pole, or some object including a device for determining orientation. Additionally, orientation can be expressed in a variety of ways, including as an azimuth (e.g., one hundred and ten degrees), a direction (e.g., a cardinal direction of a compass), or as a clock position (e.g., twelve o'clock indicating north, six o'clock indicating south, three o'clock indicating east, and nine o'clock indicating west).

[0066] The position data can be sent to, accessed by, and / or used by one or more augmented reality applications associated with the mobile computing device. For example, the augmented reality application can be a computing program implemented on the mobile computing device. In addition, the position of the mobile computing device can be used to establish one or more positions of one or more augmented reality objects relative to the mobile computing device. For example, the position of the mobile computing device can be used to establish the position of the mobile computing device (e.g., a smart phone) relative to one or more three-dimensional animated characters generated by the augmented reality application. The position data can also be used by other mapping or navigation systems, software, and / or applications.

[0067] The systems, methods, devices, apparatuses, and tangible, non-transitory computer-readable media of the disclosed technology can provide various technical effects and benefits, including improved determination of the orientation of a device. Specifically, the disclosed technology can assist a user (e.g., a user of a positioning device) in performing technical tasks based in part on the user's approximate location and imagery associated with objects visible in the field of view of an image capture component of the device, with the aid of a continuous and / or guided human-machine interaction process in which the orientation of the device is provided to the user. In addition, the disclosed technology can also provide benefits, including: improved navigation safety; reduced energy usage and data consumption of positioning devices; and more efficient use of available network resources.

[0068] The disclosed technology can improve the efficiency and safety of navigation by improving the determination of device location, thereby reducing the occurrence of users getting lost due to lack of awareness of the user's location. Providing accurate location to the user can also reduce the chance that the user will travel to an unsafe location due to inaccurate or lost location information. For example, accurate location information can reduce the occurrence of users traveling to dangerous environments (e.g., areas prone to landslides or flash floods) where their safety may be compromised.

[0069] The disclosed technology can also improve the efficiency of using network resources by reducing the amount of data that a user must access due to improper positioning and associated rerouting. More efficient determination of position in a geographic area around the current location of a positioning device allows the user to be properly positioned, thereby reducing the need to repeatedly access directions from a remote source (e.g., a navigation server). Therefore, more efficient determination of position can lead to more efficient utilization of network resources.

[0070] In addition, the disclosed technology can improve the efficiency of resource consumption (e.g., reduced battery usage of a positioning device) by providing an accurate position to a user. For example, a positioning device can save battery resources by receiving accurate position information and knowing which direction to go, thereby avoiding battery consumption associated with repeatedly expending computing resources and battery power caused by backtracking and / or rerouting.

[0071] Thus, particularly in areas where external signals such as navigation satellite signals are limited or blocked (e.g., blocked by tall buildings), the disclosed technology can allow users of positioning devices to more efficiently perform the technical task of determining and providing the position of the device. Thus, the user is provided with the specific benefits of improved security, reduced network bandwidth utilization, and more efficient use of computing and battery resources. In addition, any specific benefits provided to the user can be used to improve the efficiency of a variety of devices and services including positioning devices and / or positioning services in which the position of the device is provided or requested. Thus, the improvements provided by the disclosed technology can result in tangible benefits to various devices and / or systems including mechanical, electronic, and computing systems associated with navigation and / or determination of position.

[0072] Reference now Figure 1-10 , example embodiments of the present disclosure will be discussed in more detail. Figure 1 A diagram depicting an example system according to an example embodiment of the present disclosure. System 100 includes computing device 102 , computing system 130 , training computing system 150 , and one or more remote computing systems 170 communicatively connected and / or coupled via network 180 .

[0073] The computing device 102 can include one or more processors 112 and memory 114. The one or more processors 112 can include any suitable processing device (e.g., a processor core, a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a controller, and / or a microcontroller) and can be one processor or multiple processors operatively connected. The memory 114 can include one or more non-transitory computer-readable storage media, including RAM, NVRAM, ROM, EEPROM, EPROM, flash memory devices, disks, and combinations thereof. The memory 114 can store data 116 and instructions 118 executed by the processor 112 to cause the computing device 102 to perform one or more operations. In some embodiments, the data 116 can include: location data, which can include information associated with the location of the computing device 102; image data associated with one or more images captured by an image capture device or a component of the computing device 102; data associated with inputs, outputs, and / or one or more operations of one or more machine learning models; and / or position data associated with the position of the computing device 102 and / or one or more objects external to the computing device 102. In some embodiments, computing device 102 can include any attributes and / or any capabilities of computing system 130 , and can perform one or more operations including any of the operations performed by computing system 130 .

[0074] Computing device 102 can be implemented in and / or include any type of computing device, including, for example, a positioning device (e.g., a computing device configured to perform any of the operations described herein, including one or more operations associated with navigation and / or determination of position), a personal computing device (e.g., a laptop computing device or a desktop computing device), a mobile computing device (e.g., a smart phone or a tablet), a controller, a wearable computing device (e.g., a smart watch), and / or an embedded computing device.

[0075] Additionally, computing device 102 can be configured to perform one or more operations including: capturing one or more images of an environment visible in a field of view of the mobile computing device; receiving location data associated with the location of the mobile computing device; generating image data including information associated with the one or more images; determining one or more object classifications for one or more objects in the environment based at least in part on the image data and one or more machine learning models; determining one or more corresponding positions of one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications for the one or more objects; and / or generating location data including a geographic position of the mobile computing device based at least in part on the one or more corresponding (geographic) positions of the one or more objects.

[0076] In some embodiments, computing device 102 can perform one or more operations including sending position data to one or more augmented reality applications associated with the mobile computing device.

[0077] In some implementations, the computing device 102 can implement and / or include one or more machine learning models 120. For example, the one or more machine learning models 120 can include various machine learning models, including neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear models and / or linear models. The neural network can include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory network recurrent neural network), a convolutional neural network, and / or other forms of neural networks. Figure 1-10 Let’s discuss examples of one or more machine learning models 120.

[0078] In some embodiments, the one or more machine learning models 120 can be received from a computing system 130 (e.g., a server computing system) via a network 180, stored in a computing device memory 114, and then used or otherwise implemented by one or more processors 112. In some embodiments, the computing device 102 can implement multiple parallel instances of a single machine learning model of the one or more machine learning models 120 (e.g., for determining one or more object classifications of one or more objects across multiple instances of the machine learning model 120). More specifically, the one or more machine learning models 120 can be configured and trained to perform any operations performed by the computing system 130.

[0079] Additionally, or alternatively, one or more machine learning models 140 can be included in a computing system 130 that communicates with the computing device 102, for example, according to a client-server relationship, or otherwise stored and implemented by the computing system 130. For example, the machine learning model 140 can be implemented by the computing system 130 as part of a web service (navigation and / or position determination service). Thus, one or more machine learning models 120 can be stored and implemented on the computing device 102 and / or one or more machine learning models 140 can be stored and implemented on the computing system 130.

[0080] The computing device 102 can also include one or more user input components 122 that can receive user input. For example, the user input component 122 can be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to touch (e.g., a finger and / or a stylus) of a user input. The touch-sensitive component can be used to implement a virtual keyboard. Other example user input components include a microphone, a mechanical keyboard, an electromechanical keyboard, and / or other means by which a user can provide user input.

[0081] The computing system 130 can include one or more processors 132 and memory 134. The one or more processors 132 can include any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, and / or a microcontroller) and can include one processor or multiple processors operatively connected. The memory 134 can include one or more non-transitory computer-readable storage media, including RAM, ROM, EEPROM, EPROM, flash memory devices, disks, and combinations thereof. The memory 134 can store data 136 and instructions 138 executed by the processor 132 to cause the computing system 130 to perform operations. In some embodiments, the data 136 can include: position data and / or image data.

[0082] Additionally, in some embodiments, computing system 130 can be configured to perform the operations of a server computing device, including sending and / or receiving data including position data, image data, orientation data, and / or data associated with inputs, outputs, and / or one or more operations of one or more machine learning models to and / or from one or more computing devices and / or computing systems including computing device 102, training computing system 150, and / or remote computing system 170. In some embodiments, computing system 130 can include any attributes and / or any capabilities of computing device 102 and can perform one or more operations including any operations performed by computing device 102.

[0083] In addition, computing system 130 can be implemented in any type of computing system and / or include any type of computing system, including, for example: a navigation computing system (e.g., a computing system configured to perform any operations described herein, including one or more operations associated with the following data: position data, image data, data associated with input, output and / or one or more operations of one or more machine learning models, and / or orientation data), a personal computing device (e.g., a laptop computing device or a desktop computing device), a mobile computing device (e.g., a smart phone or a tablet), a server computing system (e.g., a computing system configured to provide data including the following data: position data, image data, data associated with input, output and / or one or more operations of one or more machine learning models, and / or orientation data) and / or a controller.

[0084] In addition, server computing system 130 can be configured to perform image content analysis on one or more inputs provided to server computing system 130 (e.g., image data including one or more images). For example, server computing system 130 can receive data via network 180. The data can include image data including one or more images and / or associated metadata (e.g., the location (e.g., latitude, longitude, and / or altitude) at which the image was captured). Server computing system 130 can then perform various operations, which can include determining one or more object classifications for the one or more images using one or more machine learning models 140. As another example, server computing system 130 can use object recognition techniques to detect one or more objects in an image (e.g., a vehicle, a portion of a vehicle, a building, a path of travel, a location identifier, text, a number, and / or a natural feature including daylight) based at least in part on the recognition of the one or more objects. In another example, server computing system 130 can receive data from one or more remote computing systems (e.g., one or more remote computing systems 170), the data can include an image that has been associated with metadata (e.g., an image with user-associated metadata indicating an address associated with an image generated by a user device). The data received by server computing system 130 can then be stored (eg, in an image repository) for later use by computing system 130 .

[0085] In some implementations, computing system 130 includes and / or is otherwise implemented by one or more server computing devices. In instances where computing system 130 includes multiple server computing devices, such server computing devices can operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0086] As described above, the computing system 130 can store or otherwise include one or more machine learning models 140. For example, the one or more machine learning models 140 can include a variety of machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Figure 1-10 Let’s discuss examples of one or more machine learning models 140.

[0087] Computing device 102 and / or computing system 130 can train one or more machine learning models 120 and / or 140 via interaction with a training computing system 150, which is communicatively connected and / or coupled via network 180. Training computing system 150 can be separate from computing system 130 or can be part of computing system 130.

[0088] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, and / or a microcontroller), and can be one processor or multiple processors operatively connected. The memory 154 can include one or more non-transitory computer-readable storage media, including RAM, ROM, EEPROM, EPROM, flash memory devices, disks, and combinations thereof. The memory 154 can store data 156 and instructions 158 executed by the processor 152 to cause the training computing system 150 to perform operations. In some embodiments, the data 156 can include position data, image data, data associated with inputs, outputs, and / or one or more operations of one or more machine learning models, and / or orientation data. In some embodiments, the training computing system 150 includes one or more server computing devices or is otherwise implemented by one or more server computing devices.

[0089] The training computing system 150 can include a model trainer 160 that trains one or more machine learning models 120 and / or one or more machine learning models 140 stored in the computing device 102 and / or the computing system 130, respectively, using various training or learning techniques, including, for example, back-error propagation. In some embodiments, performing back-error propagation can include performing truncated backpropagation through time. The model trainer 160 can perform many common techniques (e.g., weight decay and / or dropout) to improve the general capabilities of the trained model.

[0090] Specifically, the model trainer 160 can train one or more machine learning models 120 and / or one or more machine learning models 140 based on a set of training data 162. The training data 162 can include, for example: any location data and / or image data containing images of various vehicles; parts of vehicles; road features; signs; and / or road signs. For example, the training data can include one or more labeled training objects that can be used to train one or more machine learning models to perform various operations, including: determining one or more object classifications of one or more objects; determining, recognizing and / or identifying one or more classifications of one or more objects; and / or estimating the orientation of an object. One or more machine learning models 120 and / or one or more machine learning models 140 can be configured and trained to perform any of the one or more operations performed by the computing device 102 and / or the computing system 130. For example, one or more machine learning models 120 can be configured and / or trained to perform various operations, including: capturing one or more images of an environment visible in a field of view of a mobile computing device; receiving location data associated with a location of the mobile computing device; generating image data including information associated with the one or more images; determining one or more object classifications of one or more objects in the environment based at least in part on the image data and the one or more machine learning models; determining one or more corresponding positions of one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications of the one or more objects; and / or generating position data including a geographic position of the mobile computing device based at least in part on one or more corresponding (geographic) positions of the one or more objects.

[0091] In some implementations, if the user has provided consent, training examples can be provided by computing device 102. Thus, in such implementations, one or more machine learning models 120 provided to computing device 102 can be trained by training computing system 150 based at least in part on user-specific data received from computing device 102, the user-specific data including: previously classified objects and / or indications that objects were misclassified.

[0092] The model trainer 160 can include computer logic that is utilized to provide the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general purpose processor. For example, in some embodiments, the model trainer 160 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other embodiments, the model trainer 160 includes one or more sets of computer executable instructions stored in a tangible computer readable storage medium including a RAM hard disk or an optical or magnetic medium.

[0093] Each of the one or more remote computing systems 170 can include one or more processors 172 and memory 174. The one or more processors 172 can include any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, and / or a microcontroller) and can include one processor or multiple processors operatively connected. The memory 174 can include one or more non-transitory computer-readable storage media, including RAM, ROM, EEPROM, EPROM, flash memory devices, disks, and combinations thereof. The memory 174 can store data 176 and instructions 178 executed by the processor 172 to cause the remote computing system 170 to perform operations.

[0094] In some embodiments, one or more remote computing systems 170 include or are otherwise implemented by one or more computing devices. In instances where one or more remote computing systems 170 include multiple computing devices, such computing devices can operate according to a sequential computing architecture, a parallel computing architecture, and / or some combination thereof. In addition, one or more remote computing systems 170 can be used to collect, generate, send, and / or receive: one or more signals and / or data including location data, which can include information associated with the location of computing device 102; image data associated with one or more images captured by an image capture device or a component of computing device 102; data associated with inputs, outputs, and / or one or more operations of one or more machine learning models; and / or location data associated with the location of computing device 102 and / or one or more objects external to computing device 102. One or more remote computing systems 170 can include a smartphone device, and a user of the smartphone device can use the smartphone device to access the location data, image data, and / or location data.

[0095] For example, a user of a smart phone device can record location data and / or image data, including latitude, longitude, and / or altitude associated with one or more images captured by the user. The information from the smart phone device can then be sent to one or more remote computing systems 170, which can include a repository that can be used to store network data including user-provided location data. In some embodiments, the data provided by the user can be strongly encrypted, anonymized (e.g., any personal information or other information associated with the user's identity is not collected or deleted after receipt), and maintained in a secure and privacy-enhancing manner for use by one or more remote computing systems 170, which can provide location data and / or image data for use with other computing devices and / or systems including those described herein.

[0096] Network 180 can include any type of communication network, including a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof and can include any number of wired or wireless links. In general, communications on network 180 can be carried via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0097] Figure 1 An example computing system that can be used to implement the present disclosure is illustrated. Other computing systems can also be used. For example, in some embodiments, computing device 102 and / or computing system 130 can include model trainer 160 and training data 162. In such embodiments, one or more machine learning models 120 can be both trained and used locally on computing device 102 and / or computing system 130. In some such embodiments, computing device 102 and / or computing system 130 can implement model trainer 160 to personalize one or more machine learning models 120 based on user-specific data.

[0098] Figure 2 A diagram depicting an example device according to an example embodiment of the present disclosure. Computing device 200 can include one or more attributes and / or capabilities of computing device 102, computing system 130, training computing system 150, and / or one or more remote computing systems 170. In addition, computing device 200 can perform one or more actions and / or operations, including: Figure 1 One or more actions and / or operations performed by computing device 102, computing system 130, training computing system 150, and / or one or more remote computing systems 170 as depicted in FIG.

[0099] like Figure 2 As shown in , the computing device 200 can include one or more storage devices 202, position data 204, image data 206, orientation data 208, one or more interconnects 210, one or more processors 220, a network interface 222, one or more mass storage devices 224, one or more output devices 226, one or more sensors 228, one or more input devices 230 and / or a positioning device 232.

[0100] The one or more storage devices 202 can store information and / or data (e.g., position data 204, image data 206, and / or orientation data 208). In addition, the one or more storage devices 202 can include one or more non-transitory computer-readable storage media, including RAM, ROM, EEPROM, EPROM, flash memory devices, disks, and combinations thereof. The information and / or data stored by the one or more storage devices 202 can be executed by the one or more processors 220 to cause the computing device 200 to perform operations.

[0101] The location data 204 can include data stored in the memory 114, the memory 134, the memory 154, and / or the memory 174, respectively (e.g., Figure 1 116, data 136, data 156, and / or data 176) and / or instructions (e.g., Figure 1 In some embodiments, the location data 204 can include one or more portions of instructions 118, instructions 138, instructions 158, and / or instructions 178 depicted in FIG. 1 . Additionally, the location data 204 can include information associated with one or more locations including the location of the computing device 200. In some embodiments, the location data 204 can include one or more portions of instructions 118, instructions 138, instructions 158, and / or instructions 178 depicted in FIG. 1 . Figure 1 The location data 204 is received by the computing system 130 depicted in and / or one or more navigation satellites (eg, GPS).

[0102] Image data 206 can include data stored in memory 114, memory 134, memory 154, and / or memory 174, respectively. Figure 1 116, data 136, data 156, and / or data 176 and / or instructions (e.g., Figure 1 In some embodiments, image data 206 can include one or more portions of instructions 118, instructions 138, instructions 158, and / or instructions 178 depicted in FIG. 1 . In addition, image data 206 can include information associated with one or more images of one or more objects in the environment visible in the field of view of computing device 200. In some embodiments, image data 206 can be obtained from one or more computing systems (e.g., Figure 1 The computing system 130 depicted in receives the image data 206.

[0103] Position data 208 can include data stored in memory 114, memory 134, memory 154, and / or memory 174, respectively. Figure 1 116, data 136, data 156, and / or data 176 and / or instructions (e.g., Figure 1In some embodiments, the position data 208 can include one or more portions of the instructions 118, instructions 138, instructions 158, and / or instructions 178 depicted in the drawings. In addition, the position data 208 can include information associated with the position (e.g., compass direction and / or azimuth) of the computing device 200. In some embodiments, the position data 208 can be obtained from one or more computing systems (e.g., Figure 1 The computing system 130 depicted in FIG. 1 receives the position data 208 .

[0104] The one or more interconnects 210 can include one or more interconnects or buses that can be used to send and / or receive one or more signals (e.g., electronic signals) and / or data (e.g., position data 204, image data 206, and / or orientation data 208) between components of the computing device 200, including one or more storage devices 202, one or more processors 220, network interface 222, one or more mass storage devices 224, one or more output devices 226, one or more sensors 228 (e.g., sensor arrays), and / or one or more input devices 230. The one or more interconnects 210 can be arranged or configured in different ways including as parallel or serial connections. In addition, the one or more interconnects 210 can include: one or more internal buses for connecting internal components of the computing device 200; and one or more external buses for connecting internal components of the computing device 200 to one or more external devices. As an example, one or more interconnects 210 can include different interfaces including Industry Standard Architecture (ISA), Extended ISA, Peripheral Component Interconnect (PCI), PCI Express, Serial AT Attachment (SATA), HyperTransport (HT), USB (Universal Serial Bus), Thunderbolt, IEEE 1394 interface (FireWire) and / or other interfaces that can be used to connect components.

[0105] The one or more processors 220 can include one or more computer processors configured to execute one or more instructions stored in the one or more storage devices 202. For example, the one or more processors 220 can include one or more general-purpose central processing units (CPUs), application-specific integrated circuits (ASICs), and / or one or more graphics processing units (GPUs). In addition, the one or more processors 220 can perform one or more actions and / or operations, including one or more actions and / or operations associated with the position data 204, the image data 206, and / or the orientation data 208. The one or more processors 220 can include single or multiple core devices, including microprocessors, microcontrollers, integrated circuits, and / or logic devices.

[0106] The network interface 222 can support network communications. For example, the network interface 222 can support communications via a network including a local area network and / or a wide area network (e.g., the Internet). One or more mass storage devices 224 (e.g., a hard drive and / or a solid state drive) can be used to store data, including position data 204, image data 206, and / or orientation data 208. One or more output devices 226 can include one or more display devices (e.g., an LCD display, an OLED display, and / or a CRT display), one or more light sources (e.g., LEDs), one or more speakers, and / or one or more tactile output devices.

[0107] The one or more input devices 230 can include one or more keyboards, one or more touch-sensitive devices (e.g., a touch screen display), one or more buttons (e.g., an ON / OFF button and / or a YES / NO button), one or more microphones, and / or one or more cameras (e.g., for capturing images of the device's environment).

[0108] The one or more storage devices 202 and the one or more mass storage devices 224 are illustrated separately, however, the one or more storage devices 202 and the one or more mass storage devices 224 can be areas within the same storage module. The computing device 200 can include one or more additional processors, storage devices, network interfaces that can be provided separately or can be provided on the same chip or board. The one or more storage devices 202 and the one or more mass storage devices 224 can include one or more computer-readable media, including but not limited to non-transitory computer-readable media, RAM, ROM, hard disk drives, flash drives and / or other storage devices.

[0109] One or more storage devices 202 can store a collection of instructions for applications including an operating system that can be associated with various software applications or data. One or more storage devices 202 can be used to operate various applications including a mobile operating system developed specifically for mobile devices. As such, one or more storage devices 202 can store instructions that allow software applications to access data including wireless network parameters (e.g., identification of the wireless network, quality of service), and call various services including (e.g., via a global positioning system (GPS) or WLAN), location determination, and / or wireless network data call source services. In other embodiments, one or more storage devices 202 can be used to operate or execute a general-purpose operating system that operates on both mobile and fixed devices (such as, for example, smart phones and desktop computers).

[0110] Software applications capable of being operated or executed by computing device 200 can include Figure 1In addition, software applications that can be operated and / or executed by the computing device 200 can include local applications and / or web-based applications.

[0111] Positioning device 232 can include one or more devices or circuits for determining the location of computing device 200. For example, positioning device 232 can determine the actual and / or relative location of computing device 200 by using a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), BeiDou Satellite Navigation and Positioning System), an inertial navigation system, a dead reckoning system based on an IP address, by using triangulation and / or proximity to cellular towers or Wi-Fi hotspots, beacons, etc., and / or other appropriate techniques for determining location.

[0112] Figure 3 An example of position determination in an environment according to an example embodiment of the present disclosure is depicted. Any operations and / or actions performed in environment 300 can be performed by a computing device and / or computing system including one or more attributes and / or capabilities of computing device 102, computing system 130, training computing system 150, and / or one or more remote computing systems 170. Figure 3 As shown in , environment 300 includes travel path 302, travel in a west direction 304, travel in an east direction 306, travel path 308, travel in a north direction 310, mobile computing device 312, vehicle 314, location 316, vehicle 318, location 320, mobile computing device 322, vehicle 324, location 326, and location identifier 328.

[0113] Environment 300 can include any geographic area or venue. In this example, environment 300 is an urban area including an intersection of travel path 302 and travel path 308. Travel path 302 is a two-way road with travel 304 on its north side along a westbound direction and travel 306 on its south side along an eastbound direction. Travel path 304 is a one-way road with travel from south to north along a northbound direction 310.

[0114] Mobile computing device 312 can include any of the mobile computing devices described herein (e.g., Figure 1 102). Additionally, the mobile computing device can be a portable device that is configured to perform one or more operations including determining a position using a combination of location data and image data as described herein. For example, the mobile computing device 312 can determine the location of the mobile computing device 312 by obtaining location data associated with one or more GPS signals from which the approximate latitude, longitude, and / or altitude of the mobile computing device 312 is derived.

[0115] Additionally, using location data associated with the location (e.g., approximate location) of mobile computing device 312, mobile computing device 312 can determine that the user is on a sidewalk adjacent to path of travel 302. The user of mobile computing device 312 can point mobile computing device 312 in front of vehicle 314, which is located on the south side of path of travel 302. Mobile computing device 312 can then determine that vehicle 314 is located on the opposite side of the street and can also use one or more machine learning models implemented on mobile computing device 312 to determine that the front of vehicle 314 is located in the field of view of the image capture component of mobile computing device 312. In some embodiments, multiple images of vehicle 314 can be used to determine the direction of travel of vehicle 314 along path of travel 302. For example, a video stream can be used to determine the direction of travel of vehicle 314 over time.

[0116] Based at least in part on the direction of travel of vehicle 312, mobile computing device 312 is able to determine that the direction of the front of vehicle 314 relative to the location of mobile computing device 312 is direction 316 that is roughly southwest with a bearing angle of approximately two hundred and twenty (220) degrees from true north.

[0117] Additionally, the user of mobile computing device 312 can point mobile computing device 312 in front of vehicle 318 that is on the north side of path of travel 302. Mobile computing device 312 can then determine that the bearing of the front of vehicle 318 relative to the location of mobile computing device 312 is bearing 320, which is roughly southeast with an azimuth of approximately one hundred and ten (110) degrees from true north.

[0118] Additionally, the user of mobile computing device 322 can point mobile computing device 312 to the rear of vehicle 324 that is located on path of travel 308. Mobile computing device 322 can then use one or more machine learning models to determine that the rear of vehicle 324 is located in the field of view of the image capture component of mobile computing device 322. Additionally, mobile computing device 322 can determine that the orientation of the rear of vehicle 324 relative to the location of mobile computing device 322 is orientation 326 that is roughly a northwest orientation having an azimuth of approximately three hundred thirty (330) degrees from true north.

[0119] In some embodiments, mobile computing device 312 and / or mobile computing device 322 can use location identifier 328 to determine their respective positions. For example, location identifier 328 can be a road sign indicating a name and number associated with travel path 302 and travel path 308. For example, location identifier 328 can include two signs: a first sign marked "400 East Street" parallel to travel path 302 and a second sign marked "South Street" parallel to travel path 308. Mobile computing device 312 and / or mobile computing device 322 can generate image data based at least in part on the capture of one or more images of location identifier 328. In addition, image data can be used as input to one or more machine learning models that have been configured and / or trained to recognize text. Based at least in part on the recognition of text on location identifier 328, mobile computing device 312 and / or mobile computing device 322 can determine the position of location identifier 328 relative to mobile computing device 312 and / or mobile computing device 322, respectively. For example, from the vantage point of mobile computing device 312 , the field of view of mobile computing device 312 can capture two landmarks of location marker 328 at different angles that can be used to determine the location of location marker 328 relative to mobile computing device 312 .

[0120] Figure 4 Depicts an example of position determination in an environment according to an example embodiment of the present disclosure. Any operations and / or actions performed in environment 400 can be performed by a computing device and / or computing system including one or more attributes and / or capabilities of computing device 102, computing system 130, training computing system 150, and / or one or more remote computing systems 170. Figure 4 As shown in , environment 400 includes mobile computing device 402 , sun 404 , building 406 , building 408 , area 410 , area 412 , and shadowed area 414 .

[0121] Environment 400 can include any geographic area or geographic location, such as an urban area including sun 404 and buildings 406. In this example, the time is three o'clock in the afternoon (3:00 p.m.), the sky is cloudless, and the sun 404 is shining brightly overhead.

[0122] Mobile computing device 402 can include a mobile computing device as described herein (e.g., Figure 1400). Additionally, mobile computing device 402 can be a portable device that is configured to perform one or more operations including determining a position using a combination of position data and image data as described herein. For example, mobile computing device 402 can determine a position of mobile computing device 402 based at least in part on image data, which can be analyzed to determine an amount of daylight in one or more portions of environment 400.

[0123] In this example, a user of mobile computing device 402 is located between building 406 and building 408. Due to the height and position of building 406, the user of mobile computing device 402 cannot directly see sun 404. As part of determining the orientation of mobile computing device 402, the user can aim an image capture component of mobile computing device 402 at area 410 (e.g., a wall) of building 408 so that area 410 is within the field of view of mobile computing device 402. Area 410 is not facing sun 404, and as a result, the light intensity (daylight) on the surface of area 410 is relatively low. The user of mobile computing device 402 can also capture image data that includes shadow area 414. Mobile computing device 402 can determine the position of the sun based on the light intensity on the surface of area 410 and / or the direction of the shadows in shadow area 414.

[0124] In contrast to area 410, if the user aims mobile computing device 402 at area 412 (e.g., a column) of building 406, the light intensity (sunlight) will be significant because area 412 faces sun 404 and is not obstructed. Based at least in part on the timing and amount of light projected on area 412, mobile computing device 402 can determine the position of sun 404. The position of sun 404 can then be used in determining the position of area 412 relative to mobile computing device 402. This can facilitate more accurate determination of position even in environments where accurate position information (e.g., GPS) may be unavailable due to interference or obstruction from buildings (e.g., such as buildings 406 and 408).

[0125] In some embodiments, the mobile computing device 402 can determine the time of day and use the time of day as a factor in determining the relative intensity of sunlight in different parts of the environment. For example, the amount of sunlight reflected by a surface is different at seven o'clock in the evening (7:00 p.m.) compared to eleven o'clock in the morning (11:00 a.m.).

[0126] Figure 5An example of position determination in an environment according to an example embodiment of the present disclosure is depicted. Navigation computing device 500 can include one or more attributes and / or capabilities of computing device 102, computing device 200, computing system 130, training computing system 150, and / or one or more remote computing systems 170. In addition, navigation computing device 500 can perform one or more actions and / or operations, including: Figure 1 One or more actions and / or operations performed by computing device 102, computing system 130, training computing system 150, and / or one or more remote computing systems 170 as depicted in FIG. Figure 5 As shown in , mobile computing device 500 includes display component 502, audio component 504, path of travel object 506, sidewalk object 508, curb object 510, vehicle object 512, rear windshield object 514, headlight object 516, license plate object 518, rear bumper object 520, tire object 522, and orientation indication 524.

[0127] The mobile computing device 500 can be configured to receive data and / or information, including location data, image data, and / or any data associated with determining the location of the mobile computing device 500 relative to one or more objects in the surrounding environment. For example, the mobile computing device 500 can be implemented as a personal computing device (e.g., a smart phone) or an in-vehicle navigation system that can receive location data and / or image data and / or perform operations on location data and / or image data, and which can include one or more machine learning models that can be configured to detect, recognize, and / or identify one or more objects in the environment. In addition, the one or more machine learning models of the mobile computing device 500 can be configured and / or trained to determine the location and / or distance of one or more objects relative to the mobile computing device.

[0128] As shown, mobile computing device 500 includes display component 502, which can display images including text, one or more instructions, one or more pictures, and / or one or more graphics. In addition, display component 502 can be configured to detect interactions with display component 502. For example, display component 502 can detect interactions using one or more sensors including one or more capacitive sensors, one or more resistive sensors, one or more light sensors, one or more pressure sensors, and / or one or more acoustic sensors. As another example, display component 502 can include a touch-sensitive display that can detect one or more touches made by a user of mobile computing device 500.

[0129] Additionally, the mobile computing device 500 can include an audio component 504 that can be configured to output one or more sounds that include an audible indication of the orientation of the mobile computing device 500. For example, the mobile computing device 500 can generate an output that announces “the device is pointing due north” to indicate that the mobile computing device 500 is aligned in the north direction.

[0130] In this example, mobile computing device 500 includes a camera assembly (not shown) that has captured an image of a portion of an environment in a field of view of the camera assembly. The portion of the environment displayed on display assembly 502 includes a plurality of objects that can be used to determine the orientation of mobile computing device 500. Mobile computing device 500 can use one or more object classifications extracted from image data associated with the image of the environment displayed on display assembly 502 to identify a path of travel object 506 (e.g., a city street) in addition to a sidewalk object 508 and a curb object 510 adjacent to the path of travel object 506. Using the location data to determine the approximate location of mobile computing device 500, mobile computing device 500 can determine the orientation of sidewalk object 508 and / or curb object 510, and thus determine the orientation of mobile computing device 500.

[0131] Furthermore, the mobile computing device 500 can use one or more object classifications extracted from the image data associated with the image of the environment to identify the vehicle object 512. Additionally, the mobile computing device 500 can determine, recognize, and / or identify one or more portions of the vehicle object 512, including the rear windshield object 514, the headlight object 516, the license plate object 518, the rear bumper object 520, and / or the tire object 522. The mobile computing device 500 can use the recognized portions of the vehicle object 512 to determine the orientation of the vehicle relative to the location of the mobile computing device 500. For example, if the mobile computing device 500 has determined using the location data that the street on which the mobile computing device 500 is located is a one-way street and that vehicles on the street are parked on one side of the street in the same direction, then the determination that the rear bumper object 520 is in the field of view of the mobile computing device 500 can be used to determine the orientation of the rear bumper object 520, the vehicle object 512 to which the rear bumper object 520 is attached, and the orientation of the mobile computing device 500.

[0132] In addition, mobile computing device 500 can generate orientation indication 524 on display component 502. In this example, orientation indication 524 indicates that the imagery depicted in display component 502 is positioned slightly northeast. Orientation indication 524 can graphically indicate orientation, for example, by superimposing orientation indication 524 on the environment depicted by display component 502.

[0133] Figure 6A flow chart depicting orientation determination according to an example embodiment of the present disclosure. One or more portions of method 600 can be performed and / or implemented on one or more computing devices or computing systems, including, for example, computing device 102, computing system 130, training computing system 150, and / or remote computing system 170. In addition, one or more portions of method 600 can be performed or implemented as an algorithm on a hardware device or system disclosed herein. Figure 6 The steps performed in a specific order are depicted for illustration and discussion purposes. Using the disclosure provided herein, those skilled in the art will understand that the individual steps of any method disclosed herein can be adapted, modified, rearranged, omitted and / or expanded without departing from the scope of the disclosure.

[0134] At 602, method 600 can include capturing one or more images (e.g., taking one or more images), which can include capturing one or more images of an environment in which a device (e.g., a mobile computing device) capturing the one or more images is present. The one or more images can include a depiction of the environment visible in a field of view of the device (e.g., a mobile computing device capturing the one or more images). For example, computing device 102 can include an image capture component (e.g., a camera) capable of capturing one or more images of the environment.

[0135] At 604, method 600 can include receiving, accessing, obtaining, and / or retrieving location data. The location data can include information associated with a location of a device (e.g., a mobile computing device). For example, computing device 102 can receive location data based at least in part on and / or including one or more GPS signals indicating a latitude, longitude, and / or altitude of computing device 102.

[0136] At 606, method 600 can include generating, creating, and / or producing image data. The image data can include information associated with one or more images and / or information including one or more images. For example, computing device 102 can generate image data based on one or more images. In addition, the image data can be formatted so that it can be provided as input for a machine learning model. In some embodiments, generating the image data can include performing one or more operations on the image data, including color correction and / or sharpening of the associated one or more images.

[0137] At 608, method 600 can include determining one or more object classifications of one or more objects in the environment. The one or more object classifications can be based at least in part on the image data and / or one or more machine learning models. For example, computing device 102 can include one or more machine learning models 120 that can be implemented on computing device 102. Additionally, one or more machine learning models 120 can be configured and / or trained to determine one or more object classifications of one or more objects related to determination of orientation. For example, one or more object classifications can include one or more visible features associated with a shape of a vehicle object, the vehicle object including specific parts of a vehicle including windows, bumpers (e.g., front and / or rear bumpers), headlights, rear lights, windshields, and / or doors.

[0138] At 610, method 600 can include determining one or more corresponding positions of one or more objects relative to a location of a device (e.g., a mobile computing device). The one or more corresponding positions of the one or more objects relative to the location of the mobile computing device can be based at least in part on the location data and / or one or more object classifications of the one or more objects. For example, computing device 102 can use the location of computing device 102 and the one or more object classifications of the one or more objects to determine that the rear of a nearby vehicle is directly facing computing device 102, and because the vehicle is oriented north (the vehicle's orientation is determined based on matching streets and street directions to the location of computing device 102), so is computing device 102.

[0139] At 612, method 600 can include generating, creating, and / or producing position data. The position data can include a geographic position of a computing device (e.g., a mobile computing device) based at least in part on one or more corresponding (geographic) positions of one or more objects. The position data can include information associated with any aspect of the position of the computing device. Additionally, the position data can include information associated with one or more indications that can be generated to provide information and / or instructions to a user regarding the position of one or more objects in an environment within a field of view of an image capture component of the computing device.

[0140] As an example, computing device 102 can generate position data including information associated with the position of one or more objects proximate to computing device 102. Computing device 102 can then generate one or more indications, including images and / or text (e.g., arrows, bounding boxes, and associated position indications) associated with the position of the one or more objects that can be displayed on a display component of computing device 102. As a further example, the position data can be accessed by computing system 130, which can perform one or more operations on the position data and / or send the position data to another system and / or device that can include computing device 102.

[0141] At 614, method 600 can include sending the position data to one or more augmented reality applications. One or more augmented reality applications can be associated with a computing device (e.g., a mobile computing device). Additionally, the position of the mobile computing device can be used to establish one or more positions of one or more augmented reality objects relative to the mobile computing device. For example, computing device 102 can send data including the position data to a locally executed application (e.g., an application running on computing device 102). The position data can provide the position of the mobile computing device (e.g., a smart phone, augmented reality glasses, and / or an augmented reality headset) for use by the augmented reality application.

[0142] Figure 7 A flow chart for determining position according to an example embodiment of the present disclosure is depicted. One or more portions of method 700 can be performed and / or implemented on one or more computing devices or computing systems, including, for example, computing device 102, computing system 130, training computing system 150, and / or remote computing system 170. In addition, one or more portions of method 700 can be performed or implemented as an algorithm on a hardware device or system disclosed herein. In some embodiments, one or more portions of method 700 can be implemented as a Figure 6 6. The method 600 depicted in FIG. Figure 7 The steps performed in a specific order are depicted for illustration and discussion purposes. Using the disclosure provided herein, those skilled in the art will understand that the individual steps of any method disclosed herein can be adapted, modified, rearranged, omitted and / or expanded without departing from the scope of the disclosure.

[0143] At 702, method 700 can include determining one or more sides of one or more objects. The one or more sides of one or more objects can be determined based at least in part on one or more object classifications of the one or more objects. For example, computing device 102 can include one or more machine learning models configured and / or trained to determine one or more sides of various classified objects. In addition, computing device 102 can receive one or more classified objects as input and generate an output including one or more sides of the objects from a perspective from which an image thereof is captured. As another example, it can be determined that a grille of a front portion of a motor vehicle is a front side of the corresponding vehicle.

[0144] At 704, method 700 can include determining one or more corresponding orientations of one or more sides of one or more objects relative to a computing device (e.g., a mobile computing device). For example, the orientation of the object relative to computing device 102 can be based at least in part on a position of computing device 102 and a side of the object facing computing device 102. If, for example, computing device 102 is facing directly behind an object, computing device 102 will face the same direction as a front of the object. However, if computing device 102 is facing directly to the left of an object, computing device 102 will be perpendicular to the direction the object is facing.

[0145] At 706, method 700 can include determining one or more headlights associated with one or more objects. The determination of the one or more headlights can be based at least in part on one or more object classifications of the one or more objects. For example, computing device 102 can include one or more machine learning models configured and / or trained for the one or more headlights. In addition, one or more machine learning models of computing device 102 can receive image data as input and generate outputs including one or more object classifications, including one or more headlights. As another example, one or more machine learning models can be configured and trained to distinguish vehicle headlights from streetlights based on specific features of the headlights (e.g., intensity of light and / or color of light). This can facilitate improved position determination even in dark or nighttime environments.

[0146] At 708, method 700 can include determining one or more respective positions of one or more vehicle lights relative to the location of a computing device (e.g., a mobile computing device). For example, computing device 102 can determine the position of the headlights relative to computing device 102 based at least in part on the shape of the headlights, which can indicate the position (e.g., angle) of the associated vehicle. Additionally, the shape of the headlights can be useful in instances where darkness has dropped and available light does not permit capturing a clear image of the environment.

[0147] Figure 8A flow chart for determining position according to an example embodiment of the present disclosure is depicted. One or more portions of method 800 can be executed and / or implemented on one or more computing devices or computing systems, including, for example, computing device 102, computing system 130, training computing system 150, and / or remote computing system 170. In addition, one or more portions of method 800 can be executed or implemented as an algorithm on a hardware device or system disclosed herein. In some embodiments, one or more portions of method 800 can be implemented as a Figure 6 6. The method 600 depicted in FIG. Figure 8 The steps performed in a specific order are depicted for the purpose of illustration and discussion. Using the disclosure provided herein, those skilled in the art will understand that the various steps of any method disclosed herein can be adapted, modified, rearranged, omitted and / or expanded without departing from the scope of the disclosure.

[0148] At 802, method 800 can include determining a path of travel associated with an environment. The determination of the path of travel can be based at least in part on one or more object classifications. For example, computing device 102 can determine that one or more object classifications are associated with a road based on a combination of the one or more object classifications.

[0149] At 804, method 800 can include determining a position of the path of travel relative to a location of a computing device (e.g., a mobile computing device). For example, computing device 102 can access location data to match the path of travel to a road located at the location of computing device 102. Using both the path of travel and the location of computing device 102, computing device 102 can then determine the position of the path of travel relative to the location of computing device 102.

[0150] At 806, method 800 can include determining one or more directions of travel for the one or more objects. The determination of the one or more directions of travel for the one or more objects can be based at least in part on one or more object classifications of the one or more objects. For example, computing device 102 can determine the one or more directions of travel based at least in part on the orientation of the forward portion of the vehicle being in the same direction (e.g., aligned in the same direction) and the movement pattern of the vehicle also being in the same direction.

[0151] At 808, method 800 can include determining one or more positions of one or more objects relative to a location of a computing device (e.g., a mobile computing device) based at least in part on one or more directions of travel of the one or more objects. For example, computing device 102 can access location data indicating the location of computing device 102 and can use the location of computing device 102 to determine that there is a path of travel in an environment visible to an image capture component of computing device 102. In addition, one or more directions of travel of one or more objects (e.g., vehicles) can be matched to a nearby path of travel. The location of computing device 102 relative to the direction of travel of the object can then be used to determine the position of computing device 102 relative to the object. This can facilitate improved position determination in urban or metropolitan environments where location information may be inaccurate.

[0152] Fig. 9 A flow chart for determining position according to an example embodiment of the present disclosure is depicted. One or more portions of method 900 can be executed and / or implemented on one or more computing devices or computing systems, including, for example, computing device 102, computing system 130, training computing system 150, and / or remote computing system 170. In addition, one or more portions of method 900 can be executed or implemented as an algorithm on a hardware device or system disclosed herein. In some embodiments, one or more portions of method 900 can be implemented as a Figure 6 6. The method 600 depicted in FIG. Fig. 9 The steps performed in a specific order are depicted for illustration and discussion purposes. Using the disclosure provided herein, those skilled in the art will understand that the various steps of any method disclosed herein can be adapted, modified, rearranged, omitted and / or expanded without departing from the scope of the disclosure.

[0153] At 902, method 900 can include determining one or more location identifiers associated with one or more objects. The determination of the one or more location identifiers can be based at least in part on one or more object classifications. For example, computing device 102 can perform one or more text recognition operations on one or more location indicators including road signs. After performing text recognition on the road sign, computing device 102 can associate the text on the road sign with an actual address.

[0154] At 904, method 900 can include determining one or more corresponding positions of the one or more location identifiers relative to a computing device (e.g., a mobile computing device). Based on the text recognition of the one or more location identifiers, computing device 102 can determine the one or more corresponding positions of the one or more location identifiers by matching an actual address indicated on the location identifier with a physical location and then determining the position of computing device 102 with respect to the physical location. Matching the physical location to the location identifier can include accessing a data set that can include a lookup table for comparing text names to corresponding physical locations.

[0155] Fig.10 A flow chart depicting position determination according to an example embodiment of the present disclosure is provided. One or more portions of method 1000 can be executed and / or implemented on one or more computing devices or computing systems, including, for example, computing device 102, computing system 130, training computing system 150, and / or remote computing system 170. In addition, one or more portions of method 1000 can be executed or implemented as an algorithm on a hardware device or system disclosed herein. In some embodiments, one or more portions of method 1000 can be implemented as a Figure 6 6. The method 600 depicted in FIG. Fig.10 The steps performed in a specific order are depicted for illustration and discussion purposes. Using the disclosure provided herein, those skilled in the art will understand that the various steps of any method disclosed herein can be adapted, modified, rearranged, omitted and / or expanded without departing from the scope of the disclosure.

[0156] At 1002, method 1000 can include determining an amount of daylight in one or more portions of an environment. Determining the amount of daylight in one or more portions of an environment can be based at least in part on one or more features of one or more objects. For example, computing device 102 can include a camera that can be used to capture one or more images of an environment. Computing device 102 can then determine one or more features of an amount of light included in each pixel of the image by performing one or more image analysis operations on the one or more images. Image analysis can include determining an intensity value or brightness value associated with each of a plurality of pixels in the one or more images.

[0157] At 1004, method 1000 can include determining one or more respective positions of one or more objects relative to a location of a computing device (e.g., a mobile computing device) based at least in part on an amount of daylight in one or more portions of an environment. As an example, computing device 102 can use the amount of daylight in one or more portions of the environment to determine a length and / or direction of a shadow near one or more objects in the environment. The shadow can be used to determine an east-west position, which in turn can be used to determine one or more respective positions of an object casting a shadow relative to computing device 102. This can facilitate improved position determination in environments where there are few recognizable objects.

[0158] The techniques discussed herein refer to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. Those of ordinary skill in the art will recognize that the inherent flexibility of computer-based systems allows for many possible configurations, combinations, and division of tasks and functions between and among components. For example, a single server or multiple servers working in combination may be used to implement the server processing discussed herein. Databases and applications may be implemented on a single system or distributed on multiple systems. Distributed components may be operated sequentially or in parallel.

[0159] Although the subject matter has been described in detail with respect to its specific example embodiments, it will be understood that upon gaining an understanding of the above, those skilled in the art may readily generate variations, modifications, and equivalents of such embodiments. Therefore, the scope of the present disclosure is by way of example and not by way of limitation, and the subject disclosure does not exclude the inclusion of such modifications, variations, and / or additions to the subject matter that are obvious to those of ordinary skill in the art.

Claims

1. A computer-implemented method for determining a position, the computer-implemented method comprising: capturing, by a mobile computing device including one or more processors, one or more images of an environment visible in a field of view of the mobile computing device; receiving, by the mobile computing device, location data associated with a location of the mobile computing device; generating, by a mobile computing device, image data including information associated with the one or more images; determining, by a mobile computing device, one or more object classifications for one or more objects in the environment based at least in part on the image data and one or more machine learning models; determining, by the mobile computing device based at least in part on the location data and one or more object classifications for the one or more objects, one or more respective positions of the one or more objects relative to the location of the mobile computing device; as well as generating, by the mobile computing device, position data comprising a geographic position of the mobile computing device based at least in part on the one or more respective positions of the one or more objects, Wherein, determining, by the mobile computing device, one or more corresponding positions of the one or more objects relative to the position of the mobile computing device based at least in part on the position data and one or more object classifications of the one or more objects comprises: determining, by the mobile computing device, one or more directions of travel of the one or more objects based at least in part on the one or more object classifications; and One or more positions of the one or more objects relative to the location of the mobile computing device are determined, by the mobile computing device, based at least in part on the one or more directions of travel of the one or more objects.

2. The computer-implemented method of claim 1 , wherein: Determining, by the mobile computing device based at least in part on the location data and one or more object classifications for the one or more objects relative to the location of the mobile computing device, one or more respective positions of the one or more objects comprises: determining, by the mobile computing device, one or more sides of one or more objects based at least in part on the one or more object classifications; and One or more respective positions of the one or more sides of the one or more objects relative to the mobile computing device are determined by the mobile computing device.

3. The computer-implemented method of claim 2, wherein: The one or more sides of the one or more objects include one or more sides of one or more vehicles.

4. The computer-implemented method of claim 1 , wherein: The one or more object classifications are associated with one or more vehicles, and wherein the one or more object classifications include one or more headlight features, one or more taillight features, one or more front windshield features, one or more rear windshield features, one or more front grille features, one or more wheel features, one or more side window features, one or more door features, or one or more bumper features.

5. The computer-implemented method of claim 1 , wherein: Determining, by the mobile computing device, one or more respective positions of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications of the one or more objects comprises: determining, by the mobile computing device, one or more vehicle lights associated with the one or more objects based at least in part on the one or more object classifications; and One or more respective positions of the one or more vehicle lights relative to the position of the mobile computing device are determined by the mobile computing device.

6. The computer-implemented method of claim 1 , wherein: Determining, by the mobile computing device, one or more respective positions of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications of the one or more objects comprises: determining, by a mobile computing device, a path of travel associated with the environment based at least in part on the one or more object classifications; and A position of the path of travel relative to a location of the mobile computing device is determined by the mobile computing device.

7. The computer-implemented method of claim 6, wherein: The one or more object classifications include one or more indications associated with a direction of travel along the path of travel.

8. The computer-implemented method of claim 1, wherein: Determining, by the mobile computing device, one or more respective positions of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications of the one or more objects comprises: determining, by the mobile computing device, one or more location identifiers associated with the one or more objects based at least in part on the one or more object classifications; and One or more respective positions of the one or more location identifiers relative to the mobile computing device are determined by the mobile computing device.

9. The computer-implemented method of claim 8, wherein: The position of the mobile computing device is based at least in part on one or more locations respectively associated with the one or more location identifiers.

10. The computer-implemented method of claim 8, wherein: The one or more location identifiers include one or more street numbers, one or more street names, or one or more signs associated with a geographic location.

11. The computer-implemented method of claim 1 , wherein: Determining, by the mobile computing device, one or more respective positions of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications of the one or more objects comprises: determining, by the mobile computing device, an amount of daylight at one or more portions of the environment based at least in part on the one or more characteristics of the one or more objects; and One or more respective positions of the one or more objects relative to the location of the mobile computing device are determined, by the mobile computing device, based at least in part on an amount of daylight at one or more portions of the environment.

12. The computer-implemented method of claim 11, wherein: One or more respective positions of the one or more objects relative to the location of the mobile computing device are based at least in part on a time of day when an amount of daylight at one or more portions of the environment is detected.

13. The computer-implemented method of claim 1, wherein: One or more machine learning models are trained based at least in part on training data, the training data comprising one or more images of one or more vehicles at one or more distances and one or more angles relative to an image capture device that captured the one or more images of the one or more vehicles.

14. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: capturing one or more images of an environment visible in a field of view of the mobile computing device; receiving location data associated with a location of a mobile computing device; generating image data including information associated with the one or more images; determining one or more object classifications for one or more objects in the environment based at least in part on the image data and one or more machine learning models; determining one or more respective positions of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications for the one or more objects; as well as generating position data comprising a geographic position of the mobile computing device based at least in part on one or more corresponding positions of one or more objects, Wherein, determining one or more corresponding positions of the one or more objects relative to the position of the mobile computing device based at least in part on the position data and one or more object classifications of the one or more objects comprises: determining, by the mobile computing device, one or more directions of travel of the one or more objects based at least in part on the one or more object classifications; and One or more positions of the one or more objects relative to the location of the mobile computing device are determined, by the mobile computing device, based at least in part on the one or more directions of travel of the one or more objects.

15. The one or more tangible non-transitory computer readable media of claim 14, further comprising: Position data is sent to one or more augmented reality applications associated with a mobile computing device, wherein the position of the mobile computing device is used to establish one or more locations of one or more augmented reality objects relative to the mobile computing device.

16. The one or more tangible, non-transitory computer-readable media of claim 14, wherein: The one or more machine learning models are trained based at least in part on training data associated with one or more images of one or more different geographic regions, and wherein each of the one or more geographic regions is associated with a corresponding plurality of traffic regulations.

17. A computing device comprising: one or more processors; One or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: capturing one or more images of an environment visible in a field of view of the mobile computing device; receiving location data associated with a location of a mobile computing device; generating image data including information associated with the one or more images; determining one or more object classifications for one or more objects in the environment based at least in part on the image data and the one or more machine learning models; determining one or more respective positions of the one or more objects relative to the location of the mobile computing device based at least in part on the location data and the one or more object classifications for the one or more objects; and generating position data comprising a geographic position of the mobile computing device based at least in part on one or more corresponding positions of one or more objects, Wherein, determining one or more corresponding positions of the one or more objects relative to the position of the mobile computing device based at least in part on the position data and one or more object classifications of the one or more objects comprises: determining, by the mobile computing device, one or more directions of travel of the one or more objects based at least in part on the one or more object classifications; and One or more positions of the one or more objects relative to the location of the mobile computing device are determined, by the mobile computing device, based at least in part on the one or more directions of travel of the one or more objects.

18. The computing device of claim 17, wherein: The one or more objects include one or more vehicles.

19. A computing device according to claim 17 or claim 18, wherein: The one or more machine learning models are configured to identify one or more visible features associated with the image data.

Images