Immersive digital map navigation using single hand

Abstract

To determine an orientation of a user within a digital map using an immersive view at a location of the user, a client device determines a location and orientation of a camera view of the client device. The client device then presents an immersive view of a three-dimensional (3D) map from a perspective of a virtual camera having an orientation and location that matches the orientation and location of the camera view in the client device.

Description

Technical Field

[0001] The present disclosure relates to interactive geographic applications, and more particularly, to providing an immersive view of a three-dimensional (3D) map from the same location and / or orientation as a user to determine the user's orientation within the map display. Background Art

[0002] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. The work of the inventors mentioned herein (to the extent that it is described in this background section) and aspects of the specification that may not have been considered prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present disclosure.

[0003] Various computing devices support geographic software applications for displaying interactive digital maps of geographic areas. Many map applications provide users with the ability to select the type of map information or features to view and to adjust the display of the digital map. For example, a user can select from several scales and map viewing modes, such as a basic map mode that presents a traditional road map view, a satellite mode, a street-level mode, or a three-dimensional (3D) view.

[0004] Typically, when a user launches a mapping application, the mapping application may present a window that defines a viewable portion of a digital map of a geographic area that includes the user's current location, which may be referred to as a "viewport." However, it may be difficult for the user to orient themselves within the viewport, for example, to determine their location relative to a building or point of interest (POI).

[0005] In addition, mapping applications typically provide direction, rotation, and zoom controls for positioning the viewport at a desired location. For example, these controls may be provided in the form of buttons overlaying the digital map. As another example, a mapping application operated on a device equipped with a touch screen may support user gestures, such that the user can pan on the digital map by swiping her finger in the desired direction, zoom in on an area by pinching two fingers together, zoom out on an area by spreading the two fingers apart, rotate the digital map by rotating one finger while holding the other finger still, tilt the digital map by sliding two fingers in the desired tilt direction, and so on. Summary of the Invention

[0006] To determine the user's orientation within the digital map, the user can point the client device's camera view in the direction the user is facing. The client device can then determine its position and orientation via sensor data from sensors at the client device, such as a global positioning sensor (GPS), a compass, a gyroscope, an accelerometer, a transceiver, a camera, etc. The client device then presents the 3D map with an immersive view from the perspective of a virtual camera having the same orientation and position as the client device's camera view.

[0007] In some implementations, the client device may determine the initial zoom level of the immersive view based on the distance between the user and the client device. For example, the client device may identify the user's face in the front-facing camera view and determine the distance between the user and the client device based on the size of the user's face in the camera view. For example, if the user's face occupies the entire camera view, the client device may determine that the user is closer to the client device than if the user's face occupies a small portion of the camera view.

[0008] By presenting an immersive view of a 3D map based on the orientation and / or location of the client device, users can determine their own orientation within the map to identify their location relative to a particular building or point of interest (POI). The user can then interact with the building or POI in the 3D map, for example by selecting an icon overlaying the POI to view additional information about the POI. Additionally, the user can pan or zoom the 3D map to view and interact with locations beyond the horizon.

[0009] Furthermore, immersive views may be presented without the need to determine position and / or orientation with the level of accuracy that is necessary with alternative systems, such as augmented reality systems. For example, in an augmented reality system, the position and / or orientation of the client device may need to be determined extremely accurately in order to overlay augmented reality icons on real-world objects. In contrast, the immersive view of a 3D map may be from the perspective of a virtual camera that is slightly offset from the position and / or orientation of the client device's camera view, and the user may still be able to determine their own orientation within and interact with the 3D map. In this way, immersive views may be presented more frequently than with alternative systems, and may be generated without requiring as much processing power to determine the user's precise position and / or orientation.

[0010] In some implementations, the user can also select a user control to view a two-dimensional (2D) map with a specific viewport. In order to reposition the viewport of a 2D map or the immersive view of a 3D map without using finger gestures (which typically require one hand to hold the phone and the other hand to perform gestures), the client device identifies the surface of a virtual plane for the 2D map and the surface of a virtual sphere surrounding the user for the 3D map. For example, the surface of the virtual plane can be located behind and parallel to the surface of the user interface on the client device. In other examples, the surface of the virtual plane can be tilted relative to the surface of the user interface, as described in more detail below. In other examples, the surface of the virtual plane can be aligned with the ground. The surface of the virtual sphere can be a 360-degree view around the user, where the distance between the surface and the user is fixed. In any case, the client device detects the movement of the client device relative to the surface of the virtual plane or virtual sphere in a specific direction. The client device then repositions the viewport of the digital map according to the direction of movement of the client device.

[0011] In this way, the user can reposition the viewport with a single hand by holding the client device and moving the client device in a specific direction. While the client device can reposition the viewport by moving their entire body from one location to another, the user can also reposition the viewport by simply moving their hand while the user remains in the same location or within a threshold area (e.g., within a 5-meter radius, a 10-meter radius, etc.). Therefore, the client device does not need to use positioning sensors (such as global positioning service (GPS) sensors) to detect movement of the client device, which may be inaccurate. Instead, the client device uses sensors within the client device (such as accelerometers, gyroscopes, inertial measurement units (IMUs), compasses, etc.) to detect movement.

[0012] For example, if the viewport of a 2D map is positioned so that north is at the top of the digital map, then when the user moves the client device upward relative to the surface of the virtual plane, the viewport translates the digital map toward the north. When the user moves the client device downward relative to the surface of the virtual plane, the viewport translates the digital map toward the south. When the user moves the client device left relative to the virtual plane, the viewport translates the digital map toward the west, and when the user moves the client device right relative to the virtual plane, the viewport translates the digital map toward the east. Furthermore, when the user moves the client device inward toward the surface of the virtual plane, the viewport zooms in on the area depicted in the digital map, and when the user moves the client device outward away from the surface of the virtual plane, the viewport zooms out on the area depicted in the digital map. The user can also rotate or tilt the viewport by rotating or tilting the client device.

[0013] In another example, in the immersive view, when the user moves the client device upward relative to the surface of the virtual sphere, the viewport pans upward toward the sky, the top of a building, and so on. When the user moves the client device downward relative to the surface of the virtual sphere, the viewport pans the immersive view downward toward the ground. When the user moves the client device left relative to the virtual sphere and the user is facing north, the viewport pans the immersive view northwest, and when the user moves the client device right relative to the virtual sphere, the viewport pans the immersive view northeast. Additionally, when the user moves the client device inward toward the surface of the virtual sphere, the viewport zooms in on the area depicted within the immersive view, and when the user moves the client device outward away from the surface of the virtual sphere, the viewport zooms out on the area depicted within the immersive view. For example, a user can pan the client device to focus on a specific building within the immersive view. The user can then move the client device inward to view the specific building at a larger scale.

[0014] In some implementations, a user can provide a user control to lock the position of the viewport or the position of the virtual camera for the immersive view as the user moves the client device. For example, the user can lock the position of the viewport or the immersive view by pressing a finger on the user interface and holding the finger on the user interface to hold the viewport or immersive view in a locked position (e.g., a press and hold gesture), such that the position of the viewport or the position of the virtual camera for the immersive view does not change while the user's finger remains in contact with the user interface. Then, when the user releases their finger, the position of the viewport or the position of the virtual camera for the immersive view can be unlocked and can continue to change based on the movement of the client device.

[0015] In this way, the user can move the client device without panning, zooming, rotating, or tilting the viewport or immersive view. For example, the user can move the client device to the right to pan the digital map eastward. If the user wants to view an area further east but cannot move their hand further to the right, the user can lock the position of the viewport and move their hand back to the left, toward their body. The user can then release the locked position and continue to move their hand to the right to pan further east. Advantageously, this allows the user to easily pan on the digital map while minimizing movement of the user's hand and / or body. In other implementations, the viewport or immersive view can default to a locked position. The user can unlock the position of the viewport or immersive view by pressing and holding on the user interface and moving the client device while the user holds their finger on the user interface.

[0016] By repositioning the viewport of the digital map without the need for finger gestures, mapping applications improve the user experience for users, making it easier to manipulate the digital map display. Users no longer have to use both hands, freeing their other hand for additional activities, such as holding or grasping other objects. Additionally, mapping applications improve driver distraction by reducing the amount of attention required to manipulate the digital map display. For example, when a driver is viewing a digital map for directions, they no longer need to remove their hands from the steering wheel or their attention from the road to perform multi-finger gestures to manipulate the digital map. Furthermore, mapping applications improve usability for people with disabilities, arthritis, or other neurological or neuromuscular conditions that affect their ability to control their fingers or use their hands.

[0017] One embodiment of these techniques is a method for determining a user's orientation within a digital map using an immersive view at the user's location. The method includes determining a location of a client device and determining an orientation of the client device. Additionally, the method includes presenting an immersive view of a three-dimensional (3D) map from the perspective of a virtual camera having an orientation and position that matches the orientation and position of a camera view in the client device.

[0018] Another embodiment of these techniques is a client device for determining a user's orientation within a digital map using an immersive view at the user's location. The client device includes a user interface, one or more processors coupled to the user interface, and a computer-readable medium storing instructions. The computer-readable medium may be non-transitory. When executed on the one or more processors, the instructions cause the client device to determine the location of the client device and determine the orientation of the client device. The instructions further cause the client device to present an immersive view of a three-dimensional (3D) map from the perspective of a virtual camera via the user interface, the virtual camera having an orientation and position that matches the orientation and position of a camera view in the client device.

[0019] Another embodiment of these techniques is a method for generating an immersive view at a user's location to determine the user's orientation within a digital map. The method includes obtaining sensor data from one or more sensors in a client device and determining the orientation and position of the client device based on the sensor data. The method also includes generating an immersive view of a three-dimensional (3D) map from the perspective of a virtual camera, the virtual camera having a position and location that matches the orientation and location of a camera view in the client device; and providing the immersive view for display via the client device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1is a block diagram of an example system in which the techniques of this disclosure for determining a user's orientation in a map display may be implemented;

[0021] Figure 2 An example scenario is shown in which a user points a camera view of the user's client device at a real-world image;

[0022] Figure 3 Another example scenario is shown in which a user points a camera view of the user's client device at a real-world image, and the client device presents an immersive view of a 3D map with interactive controls from the same perspective as the camera view;

[0023] Figure 4 An example scenario is shown in which a user moves a client device upward toward the sky, and in response, an immersive view pans upward to reveal the tops of buildings;

[0024] Figure 5 Another example scenario is shown in which a user moves a client device inward toward a building, and in response, the immersive view zooms in on the building to show the building at a larger scale;

[0025] Figure 6 An example scenario is shown in which a client device presents a two-dimensional (2D) map with a particular viewport defining a viewable portion of the 2D map;

[0026] Figure 7 Another example scenario is shown in which the user moves the client device to the right, and in response, the viewport pans to the east;

[0027] Figure 8 An example scenario is shown in which a user rotates the client device counterclockwise, and in response, the viewport rotates so that the top of the digital map faces northwest rather than north;

[0028] Figure 9 Another example scenario is shown in which a user moves a client device toward a surface of a virtual plane and inwardly away from the user, and in response, the viewport zooms in;

[0029] Figure 10 Yet another example scenario is shown in which a user moves a client device away from a surface of a virtual plane and outward toward the user, and in response, the viewport zooms out;

[0030] Figure 11 An example scenario is shown in which a user rotates a client device toward the user while locking the position of a viewport, and in response, a virtual plane tilts relative to a surface of the client device;

[0031] Figure 12 is a flow chart of an example method for determining a user's orientation within a digital map using an immersive view at the user's location, which method may be implemented by a client device; and

[0032] Figure 13 is a flow chart of an example method for generating an immersive view at a user's location to determine the user's orientation within a digital map, which may be implemented by a server device. DETAILED DESCRIPTION

[0033] Overview

[0034] In general, techniques for determining a user's orientation within a digital map can be implemented in one or more client devices, one or more network servers, or a system comprising a combination of these. However, for clarity, the following examples primarily focus on an embodiment in which a user positions the client device's camera view to face the direction the user is facing. A mapping application at the client device obtains sensor data from sensors in the client device, such as a positioning sensor (e.g., a Global Positioning System (GPS) sensor), a compass, a gyroscope, an accelerometer, a magnetometer, a transceiver that receives wireless signals from nearby devices, a rear-facing camera that captures an image of the current rear-facing camera view, a front-facing camera that captures an image of the current front-facing camera view, or any other suitable sensor within the client device. The mapping application can then analyze the sensor data to determine the client device's position and orientation, and more specifically, the orientation of the client device's rear-facing camera view. The mapping application then transmits a request to a server device requesting an immersive view of a 3D map from the same perspective as the client device. In other implementations, the mapping application transmits the sensor data to the server device, and the server device analyzes the sensor data to determine the client device's position and orientation. In still other implementations, the client device does not obtain sensor data from a camera, and uses other sensor data from other sensors in the client device to determine the client device's rear-facing camera pose.

[0035] In any case, the server device can generate an immersive view of the 3D map by obtaining 3D map data from a map database, the 3D map data having a virtual camera pose that matches the pose of the rear camera of the client device. The server device can also select an initial zoom level for the immersive view based on the distance from the user to the front camera. For example, the client device can determine the distance from the user to the front camera based on the size of the user's face in the front camera view by analyzing an image of the current front camera view. In other implementations, the client device can transmit the image of the current front camera view to the server device, and the server device can determine the distance from the user to the front camera by analyzing the image of the current front camera view.

[0036] In any case, the server device then transmits the immersive view of the 3D map and / or the 3D map data of the immersive view to the client device for display via the mapping application. In some implementations, the server device transmits additional immersive views at different zoom levels or orientations. In this way, the user can pan or zoom the 3D map without the mapping application making additional server requests. The client device can then present the immersive view for the user to interact with the 3D map, which represents the user's current location and orientation.

[0037] As used herein, the term "pose" may refer to the position and / or orientation of an object. For example, the pose of a camera may be a combination of the camera's current position and the direction the camera is facing, the camera's pose may be the camera's current position, or the camera's pose may be the direction the camera is facing.

[0038] Example hardware and software components

[0039] Figure 1 , an example communication system 100 in which a map display direction-finding system may be implemented is shown. Communication system 100 includes a client device 10 configured to execute a geographic application 122, which may also be referred to as a "mapping application 122." Depending on the implementation, application 122 may display interactive digital maps, request and receive route data to provide driving, walking, or other navigation directions (including audio navigation directions), provide various geo-location content, and the like. Client device 10 may be operated by a user to display 2D or 3D digital maps when navigating to various locations.

[0040] In addition to the client device 10, the communication system 100 also includes a server device 60 configured to provide an immersive view and / or viewport to the client device 10. The server device 60 can be communicatively coupled to a database 80, which, in an example implementation, stores real-world images of various geographic areas. In this way, the server device 60 can compare the real-world image of the camera view from the client device 10 with the real-world images in the database 80 to identify the user's pose. The database 80 can also store 3D map data for generating an immersive view based on a specific virtual camera pose.

[0041] More generally, the server device 60 may communicate with one or more databases that store any type of suitable geospatial information or information that can be linked to a geographic context. The communication system 100 may also include a navigation data server 34 that provides navigation instructions, such as driving, walking, cycling, or public transportation instructions. In addition, the communication system 100 may include a map data server 50 that provides 2D or 3D map data to the server device 60 for generating map displays. The devices operating in the communication system 100 may be interconnected via a communication network 30.

[0042] In various implementations, the client device 10 may be a smartphone or a tablet computer. The client device 10 may include a memory 120, one or more processors (CPU) 116, a graphics processing unit (GPU) 112, an I / O module 114 including a microphone and a speaker, a user interface (UI) 32, and one or more sensors 19, including a global positioning service (GPS) module, a compass, a gyroscope, an accelerometer, a magnetometer, a camera, a transceiver, etc. The memory 120 may be a non-transitory memory and may include one or more suitable memory modules, such as a random access memory (RAM), a read-only memory (ROM), a flash memory, other types of persistent memory, etc. The I / O module 114 may be, for example, a touch screen. In various implementations, the client device 10 may include, for example, a touch screen. Figure 1 In some embodiments, the client device 10 may include fewer components than those shown, or, conversely, include additional components. In other embodiments, the client device 10 may be any suitable portable or non-portable computing device. For example, the client device 10 may be a laptop computer, a desktop computer, a wearable device (such as a smartwatch or smart glasses), a virtual reality headset, etc.

[0043] Memory 120 stores an operating system (OS) 126, which can be any type of suitable mobile or general-purpose operating system. OS 126 can include application programming interface (API) functionality that allows applications to retrieve sensor readings. For example, a software application configured to execute on computing device 10 can include instructions to call the OS 126 API to retrieve the current location of client device 10 at that moment. The API can also return a quantitative indication of how certain the API is about the estimate (e.g., as a percentage).

[0044] The memory 120 also stores a mapping application 122 that is configured to generate interactive 2D and 3D digital maps and / or perform other geographic functions, as indicated above. The mapping application 122 can retrieve navigation instructions (including audio navigation instructions) and present the navigation instructions. The mapping application 122 can also display driving, walking, or traffic directions and generally provide functions related to geography, geolocation, navigation, etc.

[0045] The mapping application 122 may include an immersive view controller 134 that generates and transmits a request for an immersive view of the 3D map to the server device 60. The request may include an indication of the pose of the rear-facing camera view of the client device 10, or may include sensor data indicating the pose of the rear-facing camera view of the client device 10, such as a real-world image from the rear-facing camera, GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data indicating the signal strength of signals received from nearby wireless devices, etc.

[0046] The request may also include an indication of the distance between the user and the front-facing camera view of client device 10, or may include sensor data indicating the distance between the user and the front-facing camera view of client device 10, such as a real-world image from a front-facing camera. Server device 60 may then analyze the real-world image from the front-facing camera to identify the user in the front-facing camera view, for example, using facial detection technology. Server device 60 may then determine the distance from the user to client device 10 based on, for example, the size of the user's face in the front-facing camera view.

[0047] The server device 60 may then determine a pose of a virtual camera for the immersive view based on the pose of the rear-facing camera view of the client device 10. The server device 60 may also determine an initial zoom level for the immersive view based on the distance from the user to the client device 10. The server device 60 may then obtain an immersive view with a matching virtual camera pose and zoom level from the database 80 and may provide the immersive view to the immersive view controller 134 for display.

[0048] In addition to requesting and displaying an immersive view, the immersive view controller 134 also receives requests from the user to pan or zoom the immersive view or the viewport of the 2D map display, for example, when the user selects a user control to view a 2D map display. The immersive view controller 134 may generate a virtual sphere around the user, as described below with reference to Figure 2 Described in more detail. For example, the virtual sphere may have a 1 meter radius around the user.

[0049] Then, when the user moves the client device 10 in a particular direction relative to the virtual sphere, the immersive view controller 134 detects the movement via sensor data from sensors 19 (such as an accelerometer or gyroscope) of the client device 10 and pans the 3D map in the direction of movement of the client device 10. For example, when the user moves the client device 10 upward relative to the surface of the virtual sphere, the immersive view controller 134 pans the immersive view upward toward the sky, the top of a building, etc. In another example, when the user moves the client device 10 inward toward the surface of the virtual sphere, the immersive view controller 134 zooms in on the area depicted within the immersive view.

[0050] When the user selects the user control to view the 2D map display, the immersive view controller 134 may generate a virtual plane for the 2D map display, as described below with reference to Figure 6 For example, the surface of the virtual plane may be located behind and parallel to the surface of the user interface 32 on the client device 10 .

[0051] Then, when the user moves the client device 10 in a particular direction relative to the virtual plane, the immersive view controller 134 detects the movement via sensor data from the sensor 19 of the client device 10 (such as an accelerometer or gyroscope) and pans the 2D map in the direction of movement of the client device 10. For example, when the user moves the client device 10 upward relative to the surface of the virtual plane, when north is at the top of the digital map, the immersive view controller 134 pans the viewport toward north. In another example, when the user moves the client device 10 inward toward the surface of the virtual plane, the immersive view controller 134 zooms in on the area depicted within the viewport.

[0052] Please note that although Figure 1 The mapping application 122 is shown as a standalone application, but the functionality of the mapping application 122 may also be provided as an online service accessible via a web browser executing on the client device 10, as a plug-in or extension to another software application executing on the client device 10, etc. The mapping application 122 may generally be provided in different versions for different corresponding operating systems. For example, the manufacturer of the client device 10 may provide a different operating system for Android. TM The platform provides a software development kit (SDK) including a mapping application 122, TM The platform provides another SDK, etc.

[0053] In some implementations, the server device 60 includes one or more processors 62 and a memory 64. The memory 64 can be a tangible, non-transitory memory and can include any type of suitable memory module, including random access memory (RAM), read-only memory (ROM), flash memory, other types of persistent memory, etc. The memory 64 stores instructions executable on the processor 62 that constitute an immersive view generation engine 68 that can generate an immersive view of a 3D map based on a virtual camera pose that matches the pose of a camera view (e.g., a rear-facing camera view) in the client device 10.

[0054] In some implementations, the immersive view generation engine 68 receives a request for an immersive view of the 3D map from the client device 10. The request may include an indication of a pose of a rear-facing camera view of the client device 10, or may include sensor data indicating the pose of the rear-facing camera view of the client device 10, such as a real-world image from a rear-facing camera, GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data indicating signal strength of signals received from nearby wireless devices, and the like.

[0055] When a request includes sensor data indicating a pose of a rear-facing camera view, the immersive view generation engine 68 determines the pose of the rear-facing camera view based on the sensor data. For example, the immersive view generation engine 68 may determine the pose of the rear-facing camera view using a particle filter. For example, the immersive view generation engine 68 may determine a first pose with a first confidence level from a GPS sensor, a second pose with a second confidence level from an accelerometer, a third pose with a third confidence level from a gyroscope, a fourth pose with a fourth confidence level from a compass, a fifth pose with a fifth confidence level from a transceiver (e.g., where location is determined based on the signal strength of a signal transmitted by a nearby wireless beacon), and so on. The particle filter may combine the poses and confidence levels from each of these sensors to generate a pose with a lower error tolerance than the confidence levels of the individual sensors. The particle filter may combine the poses and confidence levels in any suitable manner, such as by assigning a weight to each of the poses. The particle filter may also generate a probability distribution for the poses based on their respective confidence levels (e.g., using a Gaussian distribution with a confidence level corresponding to two standard deviations). The particle filter can then combine the probability distributions of the poses using Bayesian estimation to compute a least mean square (MMS) estimate.

[0056] More specifically, the particle filter can obtain N random samples (called particles) of the probability distribution to represent the probability distribution and assign a weight to each of the N random samples. The particle filter then combines the weighted particles to determine a pose with a certain confidence level that has a lower error tolerance than the confidence level of each sensor.

[0057] Additionally, the immersive view generation engine 68 may determine the user's pose by receiving a rear-facing camera view or image from a rear-facing camera of the client device 10, wherein the rear-facing camera view depicts real-world images, such as buildings, streets, vehicles, and people within the field of view of the rear-facing camera. The immersive view generation engine 68 may then determine a pose corresponding to the rear-facing camera view.

[0058] More specifically, for example, the immersive view generation engine 68 can compare the rear-facing camera view with several template camera views stored in the database 80. Each of the template camera views or images can be stored with: an indication of the camera's viewing angle or location when the image was captured (e.g., a GPS location specifying latitude and longitude coordinates, a street address, etc.), an indication of the camera's orientation when the image was captured (e.g., the camera was facing north, indicating that the camera view depicts an area north of the viewing angle), an indication of the scale or zoom level of the camera view, and / or an indication of the geographic area corresponding to the camera view (including an indication of the precise physical location of various locations within the camera view based on the scale or zoom level). The camera's orientation can include a three-axis orientation indicating the direction the camera is facing (e.g., east, west, north, south, etc.), the camera's tilt angle (e.g., parallel to the ground), and whether the camera is in a horizontal or vertical position.

[0059] For example, for a template camera view, the camera perspective may be from the corner of Main Street and State Street, with the camera facing east. The template camera view may depict a width of 5m, a length of 7m, and a height of 3m. The immersive view generation engine 68 may then create a map of the precise location of each pixel or group of pixels in the template camera view based on the camera perspective, camera orientation, and size of the template camera view. For example, if the template camera view is 5m wide, the width of the image is 500 pixels, and the orientation of the template camera view is facing east and perpendicular to the ground, each pixel may represent a physical width of approximately 1 cm. The camera perspective of the template camera view may be determined based on the GPS location of the client device that captured the camera view, and the orientation may be determined based on a gyroscope and / or compass included in the client device that captured the camera view.

[0060] In any case, the immersive view generation engine 68 can compare the rear-facing camera view from the rear-facing camera of the client device 10 with the template camera view to determine the camera angle and / or orientation of the rear-facing camera view based on the comparison. For example, the immersive view generation engine 68 can use machine learning techniques such as random forests, boosting, nearest neighbors, Bayesian networks, neural networks, support vector machines, etc. to compare the rear-facing camera view from the rear-facing camera of the client device 10 with the template camera view.

[0061] More specifically, the immersive view generation engine 68 can identify visual features of each of the template camera views by detecting stable regions within the template camera views that are detectable regardless of blur, motion, distortion, orientation, lighting, zoom, and / or other changes in camera perspective. Stable regions can be extracted from the template camera views using scale-invariant feature transforms (SIFT), speeded-up robust features (SURF), fast retinal keypoints (FREAK), binary robust invariant scalable keypoints (BRISK), or any other suitable computer vision technique. In some embodiments, keypoints can be located at high-contrast regions of the template camera view, such as edges within the template camera view. A bounding box can be formed around the keypoints, and the portion of the template camera view created by the bounding box can be a feature.

[0062] The immersive view generation engine 68 can create numerical representations of the features to generate a template feature vector, such as the width and height of the feature, the RGB pixel values ​​of the feature, the pixel location of the center point of the feature within the image, etc. The immersive view generation engine 68 can then use the feature vector as training data to create a machine learning model. For example, if the machine learning technique is nearest neighbor, the immersive view generation engine 68 can identify visual features of the rear-facing camera view from the rear-facing camera of the client device 10 to generate a feature vector. The immersive view generation engine 68 can then compare the feature vector of the rear-facing camera view from the rear-facing camera of the client device 10 with the template feature vector to identify a template camera view having a template feature vector that is closest to the feature vector of the rear-facing camera view from the rear-facing camera of the client device 10.

[0063] The immersive view generation engine 68 may then determine the camera angle and / or orientation of the rear-facing camera view to be the camera angle and / or orientation of the identified template camera view. In yet other implementations, the immersive view generation engine 68 may determine the pose of the rear-facing camera view based on any suitable combination of sensor data and a comparison of the rear-facing camera view to the template camera view. In other implementations, the client device 10 may use the above-described techniques to determine the camera angle and / or orientation of the rear-facing camera view and provide an indication of the pose of the rear-facing camera view to the server device 60.

[0064] In addition to determining the pose of the rear-facing camera, the immersive view generation engine 68 may also determine the distance between the user and the front-facing camera by, for example, analyzing an image of the front-facing camera view from the front-facing camera. For example, the immersive view generation engine 68 may receive sensor data from the client device 10 indicating the distance between the user and the front-facing camera, such as a real-world image from the front-facing camera. The immersive view generation engine 68 may then analyze the real-world image from the front-facing camera to detect the user's face in the front-facing camera view, for example, using face detection technology.

[0065] The immersive view generation engine 68 may then determine the size of the user's face relative to the size of the front-facing camera view. For example, the immersive view generation engine 68 may identify the length of the user's face (in pixels) compared to the length of the front-facing camera view. The immersive view generation engine 68 may then determine the distance from the user to the client device 10 based on the size of the user's face relative to the size of the front-facing camera view. For example, the size of the user's face relative to the size of the front-facing camera view may be inversely proportional to the distance from the user to the client device 10. The immersive view generation engine 68 may then determine an initial zoom level for the immersive view based on the distance from the user to the client device 10. As the distance between the user and the client device 10 increases, the zoom level decreases. In other implementations, the client device 10 may use the above-described techniques to determine the distance from the user to the client device 10 and provide an indication of the distance to the server device 60.

[0066] The immersive view generation engine 68 may then generate an immersive view with a virtual camera pose that matches the pose of the rear-facing camera view and an initial zoom level corresponding to the identified zoom level. The immersive view generation engine 68 may then transmit the immersive view to the client device 10 for display. In some implementations, the immersive view generation engine 68 may compare the generated immersive view with a rear-facing camera view from the rear-facing camera of the client device 10. The immersive view generation engine 68 may determine a difference metric between the generated immersive view and the rear-facing camera view, for example, based on pixel differences between the two views. When the difference metric is below a threshold difference, the immersive view generation engine 68 may then determine that the generated immersive view is from the same pose as the rear-facing camera view. In response to determining that the two views are from the same pose, the immersive view generation engine 68 may transmit the immersive view to the client device 10 for display. Otherwise, if the difference is greater than the threshold difference, the immersive view generation engine 68 may modify or change the immersive view. For example, the immersive view generation engine 68 may obtain a different immersive view from the database 80.

[0067] The immersive view generation engine 68 and the immersive view controller 134 can operate as components of a map display orientation system. Alternatively, the map display orientation system can include only server-side components and simply provide the immersive view to the immersive view controller 134 for presentation. In other words, in these embodiments, the map display orientation technology can be implemented transparently to the immersive view controller 134. Alternatively, the entire functionality of the immersive view generation engine 68 can be implemented in the immersive view controller 134.

[0068] To keep it simple, Figure 1 Server device 60 is shown as just one example of a server. However, according to some implementations, server device 60 comprises a group of one or more server devices, each equipped with one or more processors and capable of operating independently of the other server devices. Server devices operating in such a group may process requests from client devices 10 individually (e.g., based on availability); process these requests in a distributed manner, where one operation associated with processing a request is performed on one server device, while another operation associated with processing the same request is performed on another server device; or process these requests according to any other suitable technique. For this discussion, the term "server device" may refer to a single server device or a group of two or more server devices.

[0069] Sample User Interface

[0070] Figure 2An example scenario is shown in which a user 202 directs the camera view of the user's client device 10 toward a real-world image 200. The client device 10 may include a front-facing camera 204 and a rear-facing camera 206. The client device 10 includes sensors 19, such as an accelerometer and a gyroscope, to detect movement of the client device 10 in the X, Y, and Z directions, where the X direction is left and right relative to the user 202, the Y direction is up and down relative to the user 202, and the Z direction is toward and away from the user 202. The immersive view controller 134 may generate a virtual sphere around the user 202, for example, with a radius that matches the distance d from the user 202 to the client device 10. The immersive view controller 134 may then detect the motion relative to the surface of the virtual sphere via the sensors 19 and may adjust the virtual camera pose and / or zoom level of the immersive view accordingly.

[0071] When the user 202 points the camera view of the rear-facing camera 206 toward the real-world image 200, the user can select a user control via the mapping application 122 to view an immersive view of the 3D map with a virtual camera pose that matches the camera pose of the rear-facing camera 206. In some implementations, the mapping application 122 automatically generates the immersive view when the user 202 activates the rear-facing camera 206 within the mapping application 122.

[0072] In any case, in response to receiving a selection of a user control, or the user 202 activating the rear-facing camera 206 within the mapping application 122 , the immersive view controller 134 obtains sensor data from the client device sensors 19 , such as a real-world image from the rear-facing camera 206 , GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data indicating the signal strength of signals received from nearby wireless devices, etc. The immersive view controller 134 then determines the pose of the rear-facing camera 206 by analyzing the sensor data using the techniques described above, or sends the sensor data to the server device 60 to determine the pose of the rear-facing camera 206 .

[0073] The immersive view controller 134 also obtains sensor data for determining the distance d from the user 202 to the client device 10, such as a real-world image from the front-facing camera 206. The immersive view controller 134 then determines the distance d from the user 202 to the client device 10 by analyzing the sensor data using the techniques described above, or sends the sensor data to the server device 60 to determine the distance d from the user 202 to the client device 10.

[0074] Server device 60 then generates an immersive view and provides the immersive view for display via user interface 32 of client device 10 . Figure 3 An example immersive view 350 of a 3D map with interactive controls 352 is shown, where the immersive view 350 is from the same pose as the camera view of the real-world image 300 in front of the rear-facing camera 206. Figure 3 As shown, the immersive view 350 displays images of buildings and streets that are similar to the real-world buildings and streets in front of the user. This allows the user to interact with the 3D map of the area directly in front of the user. For example, the user can select a user control, such as an icon 352 overlaying a building, which, when selected, can provide additional information about the building. The user can also pan or zoom the immersive view to view distant locations or to view locations at a larger scale than the user can see real-world locations. In addition, if the user's view is blocked by a wall or building, the user can be able to "see through" the wall by viewing the immersive view 350 of the 3D map from the same posture as the view of the rear camera 206.

[0075] To pan or zoom the 3D map to obtain an immersive view at other virtual camera poses and / or zoom levels, the user may move the client device 10 in the X, Y, or Z direction. Figure 4 An example immersive view 450 is shown that is generated in response to a user moving the client device 10 upward in the Y direction. In response, the client device 10 adjusts the immersive view 450 to display images of buildings and other locations above the location displayed in the immersive view 350. The immersive view 450 includes an image depicting the top of the Empire State Building with a label for the Empire State Building and a user control 452 for providing additional information about the Empire State Building. In some implementations, when the user pans the 3D map, the immersive view controller 134 sends a request to the server device 60 for an immersive view that assumes the panned virtual camera pose. In other implementations, the immersive view controller 134 stores additional 3D map data for locations surrounding the area of ​​the initial immersive view and uses the stored 3D map data to generate a new immersive view.

[0076] Figure 5An example immersive view 550 is shown, generated in response to a user moving client device 10 away from the user in the Z direction and inward relative to the surface of the virtual sphere. In response, client device 10 adjusts the 3D map to display the Empire State Building in immersive view 550 at a larger scale than in immersive view 450. In some implementations, when the user zooms in or out of the 3D map, immersive view controller 134 sends a request to server device 60 for an immersive view at the adjusted zoom level. For example, a user may request to zoom the 3D map from a first zoom level to a second zoom level. Accordingly, immersive view controller 134 sends a request to server device 60 for an immersive view at the second zoom level. Server device 60 may generate a new immersive view of the 3D map for the second zoom level and provide the new immersive view to immersive view controller 134 for display via client device 10. In other implementations, immersive view controller 134 stores additional 3D map data for multiple zoom levels of the area of ​​the initial immersive view and uses the stored 3D map data to generate the new immersive view.

[0077] By using an immersive view generated based on the stored 3D map data, the visual quality of objects in the map (especially distant objects) can be improved, as compared to a real-world map. This is because the quality of the rear camera 206 of the client device 10 does not need to be related to the quality of the immersive view, and therefore high-quality detailed images can be consistently provided based on the 3D map data. In addition, by using the stored 3D map data to generate a new immersive view at a new zoom level, the visual detail of the immersive view can be further improved, as compared to a real-world map, for which the zoom quality may be severely limited by the quality of the rear camera 206 and / or other imaging devices, components or processes of the client device 10. Therefore, regardless of the zoom level and camera quality, the immersive view provides a consistent and improved detailed map view.

[0078] In some implementations, the user can select a user control to lock the virtual camera pose of the immersive view so that the immersive view does not change as the user moves the client device 10. For example, the user control can be a press-and-hold gesture that prevents the immersive view from changing as long as the user holds their finger on the user interface 32 of the client device 10. In other implementations, the immersive view may not change by default, and the user may have to perform a press-and-hold gesture to pan or zoom the client device 10 while moving the client device 10. By locking the virtual camera pose of the immersive view, the user can, for example, pan to the right until they can move their arm to the right. Then, when they move their arm back toward their body, the user can lock the virtual camera pose. Next, the user can unlock the virtual camera pose by, for example, releasing the press-and-hold gesture, and can continue to pan further to the right by moving their arm to the right again.

[0079] As described above, in some implementations, the mapping application 122 includes user controls to view a 2D map of an area presented in an immersive view. For example, a user can perform a specific touch gesture on the user interface 32 to transition from the immersive view to the 2D map. In other implementations, the mapping application 122 can present a 2D map with a specific viewport by default.

[0080] Figure 6 6 shows a 2D map 600 having a particular viewport 650, which is presented via the user interface 32 of the client device 10. Figure 6 As shown, the particular viewport 650 is the viewable portion of the larger 2D map 600. In some implementations, the immersive view controller 134 generates a virtual plane that can be located behind and parallel to the surface of the user interface 32 on the client device 10. Then, when the user moves the client device in a particular direction relative to the virtual plane, the immersive view controller 134 adjusts the viewport accordingly.

[0081] For example, Figure 7 An example scenario is shown in which the user moves the client device 10 to the right in the X direction. Consequently, the immersive view controller 134 adjusts the viewport 750 to present a portion of the 2D map 700 that would be east of the viewport 650 when the viewport 750 is positioned so that north is at the top of the viewport 750. Figure 8 An example scenario is shown in which the user rotates the client device 10 counterclockwise. In response, the immersive view controller 134 rotates the viewport 850 to present a portion of the 2D map 800 with the top of the viewport 850 facing northwest rather than north.

[0082] In another example, Figure 9 An example scenario is shown in which the user moves the client device 10 inwardly toward the surface of the virtual plane in the Z direction. As a result, the immersive view controller 134 zooms in on the 2D map 900 to present a viewport 950 that is magnified compared to the viewport 650. Figure 10 An example scenario is shown in which the user moves the client device 10 outward, away from the surface of the virtual plane, in the Z direction. Consequently, the immersive view controller 134 zooms out of the 2D map 1000 to present a viewport 1050 that is zoomed out compared to the viewport 650.

[0083] In another example, Figure 11 An example scenario is shown in which a user tilts client device 10 toward the user while locking the position of a virtual plane (e.g., via a user control). This user control can be the same as or different from the user control used to lock the position of the viewport. Because the position of the virtual plane is locked, the position of the surface of the virtual plane does not change. Therefore, when the user tilts client device 10 and the position of the surface of the virtual plane does not change, the surface of the virtual plane is tilted relative to client device 10. As a result, immersive view controller 134 presents a tilted view of viewport 1150, such that map features near the top of viewport 1150 appear farther away than map features near the bottom of viewport 1150. In other implementations, the user does not need to lock the position of viewport 1150 (e.g., via a user control) to tilt viewport 1150 relative to the surface of the virtual plane. Instead, immersive view controller 134 locks the position of the virtual plane by default and tilts viewport 1150 in response to the user tilting client device 10. In still other implementations, the surface of the virtual plane can remain parallel to the ground regardless of the tilt angle of client device 10. In these implementations, the viewport does not tilt.

[0084] Example method for determining a user's orientation in a map display

[0085] Figure 12 A flow chart of an example method 1200 for determining a user's orientation within a digital map using an immersive view at the user's location is shown. The method may be implemented using a set of instructions stored on a computer-readable memory and executable by one or more processors of a client device 10. For example, the method may be implemented by an immersive view controller 134 within a mapping application 122.

[0086] At block 1202, the immersive view controller 134 determines the location of the client device 10. The immersive view controller 134 may also determine the orientation of the client device 10. More specifically, the immersive view controller 134 may determine the location and orientation of the rear-facing camera of the client device 10 (the "rear-facing camera pose"). The immersive view controller 134 may determine the location and orientation by obtaining sensor data from sensors 19 in the client device 10, such as a real-world image from the rear-facing camera, GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data indicating the signal strength of signals received from nearby wireless devices, and the like. The immersive view controller 134 may then analyze the sensor data to determine the rear-facing camera pose by, for example, comparing the real-world image from the rear-facing camera to a template camera view stored in the database 80, and / or by analyzing the GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data, and the like to determine the rear-facing camera pose.

[0087] In other implementations, the immersive view controller 134 may transmit the sensor data to the server device 60 so that the server device analyzes the sensor data to determine the rear camera pose. The server device 60 may then provide an indication of the rear camera pose to the client device 10.

[0088] At block 1206, immersive view controller 134 presents an immersive view of the 3D map from the perspective of a virtual camera having an orientation and position that matches the orientation and position of the rear-facing camera view in client device 10. Immersive view controller 134 may transmit a request for the immersive view of the 3D map to server device 60. The request may include an indication of the pose of the rear-facing camera view of client device 10, or may include sensor data indicating the pose of the rear-facing camera view of client device 10, such as a real-world image from the rear-facing camera, GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data indicating the signal strength of signals received from nearby wireless devices, and the like.

[0089] The request may also include an indication of the distance between the user and the front camera view of client device 10, or may include sensor data, such as a real-world image from a front camera, indicating the distance between the user and the front camera view of client device 10. Server device 60 may then analyze the real-world image from the front camera to identify the user in the front camera view, for example, using face detection technology, and determine the distance from the user to client device 10 based on, for example, the size of the user's face in the front camera view.

[0090] The server device 60 may then determine a pose of a virtual camera for the immersive view based on the pose of the rear-facing camera view of the client device 10. The server device 60 may also determine an initial zoom level for the immersive view based on the distance from the user to the client device 10. The server device 60 may then obtain an immersive view with a matching virtual camera pose and zoom level from the database 80 and may provide the immersive view to the immersive view controller 134 for display.

[0091] Figure 13 A flow chart of an example method 1300 for generating an immersive view at a user's location to determine the user's orientation within a digital map is shown. The method may be implemented using an instruction set stored on a computer-readable memory and executable by one or more processors of a server device 60. For example, the method may be implemented by the immersive view generation engine 68.

[0092] At block 1302, the immersive view generation engine 68 obtains sensor data from the client device 10. The sensor data may include real-world images from a rear-facing camera, GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data indicating signal strength of signals received from nearby wireless devices, etc. The sensor data may be obtained as part of a request for an immersive view of a 3D map from the perspective of the client device 10.

[0093] Then, at block 1304, the immersive view generation engine 68 determines the position and orientation of the client device 10 based on the sensor data. For example, the immersive view generation engine 68 may compare the real-world image from the rear-facing camera with template camera views stored in the database 80. Additionally or alternatively, the immersive view generation engine 68 may analyze GPS data, accelerometer data, gyroscope data, compass data, magnetometer data, transceiver data, etc. to determine the position and orientation of the client device 10.

[0094] At block 1306, immersive view generation engine 68 generates an immersive view of the 3D map from the perspective of a virtual camera having an orientation and position that matches the orientation and position of the rear-facing camera view in client device 10. For example, immersive view generation engine 68 may generate the immersive view by obtaining 3D map data from database 80, the 3D map data having a virtual camera pose that matches the pose of the rear-facing camera of client device 10. Then, at block 1308, immersive view generation engine 68 provides the immersive view to client device 10 for display. In some implementations, immersive view generation engine 68 generates multiple immersive views for different zoom levels or virtual camera poses around the pose of the rear-facing camera of client device 10. Immersive view generation engine 68 provides these multiple immersive views to client device 10, allowing the user to pan or zoom the 3D map without requiring client device 10 to issue additional requests to server device 60 for immersive views within the same general area.

[0095] Additional Notes

[0096] The following additional considerations apply to the foregoing discussion. Throughout this specification, plural instances may implement components, operations, or structures described as single instances. Although the individual operations of one or more methods are shown and described as separate operations, one or more of the individual operations may be performed simultaneously, and the operations are not required to be performed in the order shown. The structures and functionality presented as separate components in the example configurations may be implemented as combined structures or components. Similarly, the structures and functionality presented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0097] Certain embodiments are described herein as including logic or multiple components, modules, or mechanisms. A module may constitute a software module (e.g., code embodied on a machine-readable medium or in a transmission signal) or a hardware module. A hardware module is a tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. In an example embodiment, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured as a hardware module by software (e.g., an application or application portion), and the hardware module operates to perform certain operations as described herein.

[0098] Unless expressly stated otherwise, discussions herein using words such as "process," "compute," "calculate," "determine," "present," "display," etc. may refer to the actions or processes of a machine (e.g., a computer) that manipulate or transform data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0099] As used herein, any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0100] Some embodiments may be described using the expressions "coupled" and "connected," as well as their derivatives. For example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact. However, the term "coupled" may also refer to two or more elements that are not in direct contact with each other, but still cooperate or interact with each other. The embodiments are not limited in this context.

[0101] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless expressly stated to the contrary, "or" refers to an inclusive or and not an exclusive or. For example, a condition A or B is satisfied by any of the following: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exists).

[0102] In addition, the use of "a" or "an" is used to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the various embodiments. The description should be read to include one or at least one, and the singular also includes the plural unless it is obvious that it is intended otherwise.

[0103] After reading this disclosure, a person of ordinary skill in the art will appreciate additional alternative structural and functional designs for determining a user's orientation within a map display using the principles disclosed herein. Thus, while specific embodiments and applications have been shown and described, it should be understood that the disclosed embodiments are not limited to the precise configurations and components disclosed herein. Various modifications, changes, and variations that will be apparent to those skilled in the art may be made to the arrangement, operation, and details of the methods and apparatus disclosed herein without departing from the spirit and scope of the appended claims.

Claims

1. A method for determining an orientation of a user within a digital map using an immersive view at the user's location, the method comprising: determining, by one or more processors in a client device, a location of the client device; determining, by the one or more processors, an orientation of the client device; as well as An immersive view of a three-dimensional (3D) map is presented by the one or more processors from the perspective of a virtual camera having an orientation and position that matches an orientation and position of a camera view in the client device. 2 . The method of claim 1 , wherein determining the orientation of the client device comprises determining an orientation of a rear-facing camera view of the client device.

3. The method according to claim 1 or 2, further comprising: An initial zoom level for presenting the immersive view is determined, by the one or more processors, based on a distance between a user and the client device.

4. The method of claim 3, further comprising: The distance between the user and the client device is determined, by the one or more processors, based on a size of the user's face in a front-facing camera view of the client device.

5. The method of any preceding claim, wherein determining the position and the orientation of the client device comprises: receiving, at the one or more processors, sensor data at the client device from one or more sensors, the sensor comprising at least one of: a positioning sensor, a compass, a gyroscope, an accelerometer, a transceiver, or a camera; and The sensor data is used, by the one or more processors, to determine the position and the orientation of the client device.

6. The method of claim 5, wherein using the sensor data to determine the position and the orientation of the client device comprises: The one or more processors provide a real-world image of the camera view from the camera to a server device to compare the real-world image with a plurality of template real-world images from a plurality of positions and orientations stored in a real-world image database to identify at least one template real-world image from the plurality of template real-world images that matches the real-world image from the camera and to identify the position and the orientation of the client device based on the position and the orientation of the identified template real-world image.

7. The method of any preceding claim, further comprising: detecting, by the one or more processors, that the client device has moved in a particular direction; as well as The immersive view is repositioned, by the one or more processors, based on a direction of movement of the client device.

8. The method of claim 7, wherein repositioning the immersive view based on the direction of movement of the client device comprises: identifying the surface of a virtual sphere surrounding the user; The immersive view is translated relative to the virtual sphere along the direction of movement of the client device.

9. The method of claim 7, wherein repositioning the immersive view based on the movement direction of the client device comprises: identifying the surface of a virtual sphere surrounding the user; A zoom level of a viewport is adjusted in response to detecting that the client device has moved toward or away from the surface of the virtual sphere.

10. The method of any preceding claim, further comprising: receiving a user input requesting that a position of the virtual camera for the immersive view be locked to a current position; detecting, by the one or more processors, that the client device has moved in a particular direction; as well as The immersive view is presented, by the one or more processors, from the perspective of the virtual camera at the current location without repositioning the immersive view based on a direction of movement of the client device.

11. The method of claim 10, wherein the user input is a press-and-hold gesture on a user interface of the client device, and wherein the immersive view does not change position while the user's finger remains in contact with the user interface.

12. A client device for determining an orientation of a user within a digital map using an immersive view at the user's location, the client device comprising: user interface; one or more processors; as well as a computer-readable memory comprising computer-executable instructions that, when executed by the one or more processors, cause the client device to: determining a location of the client device; determining an orientation of the client device; as well as An immersive view of a three-dimensional (3D) map is presented via the user interface from the perspective of a virtual camera having an orientation and position that matches an orientation and position of a camera view in the client device.

13. The client device of claim 12, wherein the orientation of the client device is determined based on an orientation of a rear-facing camera view of the client device.

14. The client device of claim 12 or 13, wherein the instructions further cause the client device to: An initial zoom level for presenting the immersive view is determined based on a distance between a user and the client device.

15. The client device of claim 14, wherein the instructions further cause the client device to: The distance between the user and the client device is determined based on a size of the user's face in a front-facing camera view of the client device.

16. The client device of any one of claims 12 to 15, wherein to determine the position and the orientation of the client device, the instructions cause the client device to: receiving sensor data from one or more sensors, the sensor comprising at least one of: a positioning sensor, a compass, a gyroscope, an accelerometer, a transceiver, or a camera; and The sensor data is used to determine the position and the orientation of the client device.

17. A method for generating an immersive view at a user's location to determine the user's orientation within a digital map, the method comprising: obtaining, at the one or more processors, sensor data from one or more sensors in the client device; determining, by the one or more processors, an orientation and a position of the client device based on the sensor data; generating, by the one or more processors, an immersive view of a three-dimensional (3D) map from the perspective of a virtual camera, the virtual camera having an orientation and position that matches the orientation and position of a camera view in the client device; as well as The immersive view is provided, by the one or more processors, for display via the client device.

18. The method of claim 17, wherein the one or more sensors include at least one of: a positioning sensor, a compass, a gyroscope, an accelerometer, a transceiver, or a camera.

19. The method of claim 17 or 18, wherein determining the orientation of the client device comprises determining an orientation of a rear-facing camera view of the client device.

20. The method of any one of claims 17 to 19, further comprising: An initial zoom level of the immersive view is determined, by the one or more processors, based on a distance between a user and the client device.

21. The method of any one of claims 17 to 19, further comprising: receiving, by the one or more processors, a request to change a zoom level of the 3D map from a first zoom level to a second zoom level; generating, by the one or more processors, a new immersive view of the 3D map for the second zoom level; as well as The new immersive view is provided, by the one or more processors, for display via the client device.

Images