Content-Aware Navigation Instructions

The mapping application adjusts voice navigation instructions to align with other media/audio content, enhancing clarity and safety by ensuring they are heard clearly without competing with other audio sources.

JP7577204B2Active Publication Date: 2024-11-01GOOGLE LLC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023519747
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-10-22
Publication Date
2024-11-01
Estimated Expiration
2040-10-22

AI Technical Summary

Technical Problem

In-vehicle navigation systems often compete for the driver's attention due to multiple audio outputs, leading to a suboptimal experience where navigation instructions are not clearly heard or understood, especially when other audio content is playing.

Method used

A mapping application adjusts voice navigation instructions based on other media/audio content being played, such as pausing at natural breaks, matching language, or adjusting volume to ensure clarity and reduce distraction.

Benefits of technology

This approach improves the clarity and safety of voice navigation by ensuring instructions are easily understood, reducing the need to compete with other audio and minimizing distractions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007577204000001
    Figure 0007577204000001
  • Figure 0007577204000002
    Figure 0007577204000002
  • Figure 0007577204000003
    Figure 0007577204000003
Patent Text Reader

Abstract

To provide content-aware voice navigation instructions, a client device running a mapping application obtains one or more voice navigation instructions for traveling along a route from a start location to a destination. The client device also identifies electronic media content being played from a source different from the mapping application that is running on or in proximity to the client device. The client device determines characteristics of the electronic media content and adjusts the voice navigation instructions according to the characteristics of the electronic media content. The client device then presents the adjusted voice navigation instructions to a user.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates to content-aware navigation instructions, and more particularly, to tailoring voice navigation instructions based on other media / audio content playing in the area or on the same device. [Background technology]

[0002] The background discussion provided herein is for the purpose of generally presenting the context of the present disclosure. The work of the presently named inventors to the extent described in this background section, as well as aspects of the present specification that may not qualify as prior art at the time of filing, are not admitted expressly or impliedly as prior art to the present disclosure.

[0003] Today, software applications that run on computers, smartphones, etc., or in embedded devices rely on step-by-step navigation. Instructions Typically, a user specifies a starting point and a destination, and the software application generates a map immediately and / or as the user travels from the starting point to the destination. , instructions display or present in audio format.

[0004] These software applications provide route-based navigation. Instructions To generate a map, these systems typically use distance, street name, and building number indicators. For example, these systems can provide drivers with directions such as "Go 1 / 4 mile, then turn right onto Maple Street." Summary of the Invention [Problem to be solved by the invention]

[0005] The voice instructions provided by an in-vehicle navigation system must generally be informative and provide sufficient and timely instructions on the route to follow without unnecessarily distracting listeners, particularly the driver. In addition, since other audio is played in the vehicle simultaneously with the voice instructions, the in-vehicle navigation system must generally compete for the driver's attention. The combination of multiple audio outputs may result in a suboptimal experience where the user is unable to hear or understand the navigation instructions. Other audio in the vehicle may also provide information indicative of the driver's preferences, such as the driver's preferred language, the driver's preferred audio playback speed, the driver's preferred destination, etc. Therefore, an improved system-user voice interface that provides voice instructions that are deeply integrated with other media content played during navigation and adapted according to other audio played in the vehicle may be beneficial. [Means for solving the problem]

[0006] In some implementations, the mapping application providing voice navigation instructions may identify other media / audio content playing in the area or on the same device (e.g., in the same vehicle, in the same room, in an outdoor area within a threshold distance of the application, etc.). The mapping application may then adjust the voice navigation instructions according to the media / audio content. For example, if the media / audio content is an audiobook, podcast, music, etc., the mapping application may wait to provide the next navigation instructions until the media / audio content breaks. More specifically, the mapping application may play the next navigation instructions after the end of a segment, the end of a song, or the end of a sentence. In another example, if the media / audio content is presented in a particular language (e.g., Spanish), the mapping application may present the voice navigation instructions in that language. In yet another example, the mapping application may provide suggestions or recommendations regarding points of interest (POIs) along a route mentioned in the media / audio content.

[0007] To identify other media / audio content being played in the area, the mapping application may communicate with other applications running on the client device, for example, via an application programming interface (API). The other applications running on the client device may provide audio playback data including characteristics of the media / audio content being played by the other applications, such as the speed (e.g., speaking rate) at which the other applications are being played (e.g., 1.5x, 2x, etc.), the language of the media / audio content, the voice characteristics of the speaker in the media / audio content, the transcription of the media / audio content, the length of the media / audio content, etc. In addition, the mapping application may communicate with other devices (e.g., vehicle head units) that are playing the media / audio content, such as via a short-range communication link. The other devices may also provide audio playback data including characteristics of the media / audio content to the mapping application. Furthermore, the mapping application may identify other media / audio content by comparing the audio fingerprint of a given media / audio content to ambient audio in the surrounding area. For example, the mapping application may obtain a library of audio fingerprints corresponding to media / audio from popular music, audio books, radio shows, podcasts, etc. If the mapping application identifies a match with one of the audio fingerprints, the mapping application may determine that media / audio content is being presented in the area.

[0008] The mapping application may then identify characteristics of the media / audio content by communicating with the application or device presenting the content to receive metadata associated with the content, or by analyzing the media / audio content using speech recognition techniques.

[0009] One exemplary embodiment of the techniques of this disclosure is a method for generating content-aware navigation instructions. The method includes generating one or more voice navigation instructions for traveling along a route from an origin to a destination at a client device via a mapping application. Instructions The method includes obtaining a mapping application, identifying electronic media content being played from a source different from the mapping application, the source being executed at or proximate to the client device, and determining characteristics of the electronic media content. The method includes providing one or more voice navigation functions according to the characteristics of the electronic media content. Instructions and adjusting at least one of the adjusted voice navigation. Instructions and presenting the result to a user.

[0010] Another example embodiment of the techniques of this disclosure is a client device for generating content-aware navigation instructions. The client device includes a speaker, one or more processors, and a non-transitory computer-readable memory coupled to the one or more processors and the speaker that stores the instructions. The instructions, when executed by the one or more processors, cause the client device to generate one or more voice navigation instructions for traveling from an origin to a destination along a route via a mapping application. Instructions The instructions further cause the client device to perform operations of obtaining a mapping application, identifying electronic media content being played from a source different from the mapping application, the source being executed at or proximate to the device, and determining characteristics of the electronic media content. Instructions and providing at least one adjusted voice navigation through a speaker. Instructions is presented to the user.

[0011] Yet another example embodiment of the techniques of this disclosure is a non-transitory computer-readable memory storing instructions that, when executed by one or more processors, cause the one or more processors to generate one or more voice navigation commands for traveling from a start location to a destination along a route via a mapping application. Instructions The instructions further cause the one or more processors to perform operations of obtaining a mapping application, identifying electronic media content being played from a source different from the mapping application, the source being executed at or proximate to the client device, and determining characteristics of the electronic media content. Instructions and providing at least one adjusted voice navigation through a speaker. Instructions is presented to the user.

[0012] The method described herein improves voice navigation by taking into account the audio characteristics of other media (audio) content. Instructions The method improves the output of voice navigation. Instructions The audio navigation is easily understood by the listener following the Instructions This improved clarity and understanding means that voice navigation instructions are safer. For example, Instructions may be adjusted to match characteristics of the media content, such as language, or its output may be delayed / adjusted to occur at a time that matches a natural break in the media content, such as the end of a sentence. In another example, the media content may be paused to allow output of voice navigation instructions, thus avoiding audio overlaps. These and other examples are described below. [Brief description of the drawings]

[0013] [Figure 1]FIG. 1 illustrates an example vehicle in which the techniques of this disclosure can be used to generate content-aware voice navigation instructions. [Diagram 2] 1 is a block diagram of an example system in which techniques for generating content-aware voice navigation instructions may be implemented. [Diagram 3] 3 illustrates an example navigation instructions data table that the content-aware voice navigation generating system of FIG. 2 can utilize to tailor navigation instructions according to particular media content. [Figure 4] 1 is a flow diagram of an example method for generating content-aware voice navigation instructions that may be implemented in a client computing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Generally speaking, techniques for providing content-aware navigation instructions can be implemented in a system that includes one or more client devices, a vehicle head unit, one or more network servers, or a combination of these devices. For clarity, however, the following examples focus primarily on embodiments in which a client device running a mapping application obtains a set of voice navigation instructions for navigating a user along a route from a start location to a destination. For example, a user may receive voice navigation instructions via the mapping application from the user's current location to a destination. Instructions , and the client device may forward the request to the navigation server. The navigation server may then generate a set of voice navigation instructions and send the set of voice navigation instructions to the client device for playback to the user.

[0015] In addition, the mapping application may communicate with other applications running on the client device (e.g., via an API) or with other devices within the vicinity of the client device (e.g., a short-range communication link), such as a vehicle head unit or other client devices. The communications may include an indication of the media content being played by the other application / device and may include characteristics of the media content, such as the language of the media content, the speaking rate of the media content, a transcription of the media content, voice characteristics of the speaker of the media content, etc. In addition to communicating with other applications / devices to identify media content, the mapping application may compare ambient audio being played in the area to a set of audio fingerprints for popular media content, such as popular songs, radio shows, podcasts, etc., to identify the media content.

[0016] In either case, the mapping application analyzes the characteristics of the media content and provides voice navigation according to the media content. Instructions The characteristics of the media content may be transmitted to the server device to determine how to adjust the voice navigation. The server device then transmits the adjusted voice navigation. Instructions , or voice navigation Instructions In another implementation, the mapping application may send an indication to the client device of how to adjust playback of the media content. Instructions It may be possible to determine how to adjust

[0017] As used herein, media content may include radio programs, podcasts, audiobooks, music, advertisements, television programs, movies, videos, or any other type of media that includes an audio component. Instructions means transportation to a destination via vehicle. Transfer instructions Includes voice navigation Instructionsmay be for any suitable mode of transportation, such as walking, cycling, public transportation, etc.

[0018] Exemplary Hardware and Software Components 1, an exemplary environment 1 in which the techniques outlined above may be implemented includes a portable device 10 and a vehicle 12 having a head unit 14. The portable device 10 may be, for example, a smartphone, a tablet computer, or an in-vehicle navigation system. The portable device 10 communicates with the head unit 14 of the vehicle 12 via a communication link 16, which may be wired (e.g., Universal Serial Bus (USB)) or wireless (e.g., Bluetooth, Wi-Fi Direct). The portable device 10 may also communicate with various content providers, servers, etc. via a wireless communication network, such as a fourth or third generation cellular network (4G or 3G, respectively).

[0019] The head unit 14 may include a display 18 for presenting navigation information, such as a digital map. The display 18 is a touch screen and includes a software keyboard for entering text input, which may include names or addresses of destinations, starting points, etc. Hardware input controls 20 and 22 on the head unit 14 and steering wheel, respectively, are used for entering alphanumeric characters or for navigational input. Instructions The head unit 14 may be used to perform other functions such as to request commands, and to play back audio instructions transmitted from the portable device 10. The head unit 14 may also include audio input and output components such as a microphone 24 and a speaker 26. The speaker 26 may be used to play back audio instructions transmitted from the portable device 10.

[0020] An exemplary communication system 100 in which a content-aware voice navigation generation system may be implemented is shown in FIG. 2. The communication system 100 includes a client device 10 configured to run a geographic application 122, sometimes referred to as a "mapping application 122." Depending on the implementation, the application 122 displays an interactive digital map and provides driving, walking, or voice navigation. Instructions Other navigation including Instructions , can request and receive routing data to provide navigation, can provide various geolocation content, and the like. The client device 10 may be operated by a user (also referred to herein as a "driver") who displays digital maps while navigating to various locations. The communication system 100 also includes a vehicle head unit 14 that may communicate with the client device 10 via a short-range communication link, such as Bluetooth, Wi-Fi Direct, or the like. Additionally, the communication system 100 may include other computing devices 92 within the vicinity of the client device 10 that may communicate with the client device 10 via a short-range communication link, such as Bluetooth, Wi-Fi Direct, or the like. For example, if the client device 10 is a driver's smartphone, the other computing devices 92 may include a passenger's smartphone in the vehicle, or the driver's tablet or wearable device.

[0021] In addition to the client device 10, the communication system 100 includes a server device 60 configured to provide content-aware voice navigation instructions to the client device 10. The server device 60 can be communicatively coupled to a database 80 that stores machine learning models for tailoring voice navigation instructions to particular media / audio content in an exemplary implementation. The training data can include a set of voice navigation instructions previously provided to the user, characteristics of the media content played when the voice navigation instructions were provided, indicators of adjustments made by the user to the voice navigation instructions, and / or self-reported indicators of the user's satisfaction with the voice navigation instructions. The training data is described in more detail below with reference to FIG. 3. Additionally, the database 80 can store a set of rules for tailoring voice navigation parameters.

[0022] More generally, server device 60 may be in communication with one or several databases that store any type of suitable geospatial information or information that can be linked to a geographic context. Instructions The communication system 100 may also include a navigation data server 34 that provides map data to a server device 60 for generating map displays. Additionally, the communication system 100 may include a map data server 50 that provides map data to a server device 60 for generating map displays. The devices operating within the communication system 100 may be interconnected via a communication network 30.

[0023] In various implementations, the client device 10 may be a smartphone or a tablet computer. The client device 10 may include a memory 120, one or more processors (CPU) 116, a graphics processing unit (GPU) 112, an I / O module 14 including a microphone and a speaker, a user interface (UI) 32, and one or several sensors 19 including a global positioning service (GPU) module. The memory 120 may be a non-transitory memory and may include one or several suitable memory modules, such as random access memory (RAM), read only memory (ROM), flash memory, other types of persistent memory, etc. The I / O module 114 may be, for example, a touch screen. In various implementations, the client device 10 may include fewer components than those shown in FIG. 2, or conversely, may include additional components. In other implementations, the client device 10 may be any suitable portable or non-portable computing device. For example, the client device 10 may be a laptop computer, a desktop computer, a wearable device such as a smart watch or smart glasses, etc.

[0024] The memory 120 stores an operating system (OS) 126, which can be any type of suitable mobile or general-purpose operating system. The OS 126 can include application programming interface (API) functions that allow applications to obtain sensor readings. For example, a software application configured to run on the computing device 10 can include instructions to call the API of the OS 126 to obtain the current location of the client device 10 at that time. The API can also return a quantitative indication (e.g., a percentage) of how confident the API is of the estimate.

[0025] The memory 120 also stores a mapping application 122 configured to generate an interactive digital map and / or perform other geographic functions, as indicated above. The mapping application 122 can receive navigation instructions, voice navigation instructions, and / or voice navigation instruction parameters, and present voice navigation instructions according to the voice navigation instruction parameters. The mapping application 122 can provide navigation instructions for driving, walking, biking, or public transportation. Instructions , and generally provide geographic, geolocation, navigation, and other related functionality.

[0026] 2 illustrates the mapping application 122 as a standalone application, it should be noted that the functionality of the mapping application 122 may also be provided in the form of an online service accessible via a web browser running on the client device 10, as a plug-in or extension for another software application running on the client device 10, etc. The mapping application 122 may be provided in different versions for different respective operating systems. For example, a manufacturer of the client device 10 may provide a software development kit (SDK) that includes the mapping application 122 for the Android™ platform, another SDK for the iOS™ platform, etc.

[0027] In addition to the mapping application 122, the memory 120 stores other client applications 132 that play media / audio content, such as music applications, video applications, streaming applications, radio applications, social media applications, etc. These applications 132 may expose APIs for communicating with the mapping application 122.

[0028] In some implementations, the server device 60 includes one or more processors 62 and a memory 64. The memory 64 may be a tangible, non-transitory memory and may include any type of suitable memory module, including random access memory (RAM), read only memory (ROM), flash memory, other types of persistent memory, etc. The memory 64 may be configured to obtain characteristics of media / audio content presented within the vicinity of the client device 10 and to perform voice navigation according to the characteristics of the media / audio content. Instructions The content-aware voice navigation generator 68 stores instructions executable on the processor 62 that configure the content-aware voice navigation generator 68 to be able to tailor voice navigation instructions within a set of media / audio content. In some implementations, the content-aware voice navigation generator 68 may generate machine learning models for tailoring voice navigation instructions to specific media / audio content. The content-aware voice navigation generator 68 also generates navigation instructions for the user from an origin to a destination. Instructions The content-aware voice navigation generator 68 may then generate a voice navigation request that is then presented by the mapping application 122 via the speaker 26. Instructions and provide the set to the client device 10.

[0029] The content-aware voice navigation generator 68 and the mapping application 122 can operate as components of a content-aware voice navigation generation system. Alternatively, the content-aware voice navigation generation system can include only server-side components and simply provide instructions to the mapping application 122 for presenting voice navigation instructions. In other words, the content-aware voice navigation generation techniques in these embodiments can be implemented transparently to the mapping application 122. As another alternative, the entire functionality of the content-aware voice navigation generator 68 can be implemented in the mapping application 122.

[0030] For simplicity, FIG. 2 illustrates server device 60 as a sole instance of a server. However, server device 60 includes a group of one or more server devices, each equipped with one or more processors and capable of operating independently of the other server devices. Server devices operating within such a group may process requests from client device 10 individually (e.g., based on availability), in a distributed manner in which one operation related to processing a request is performed on one server device and another operation related to processing the same request is performed on another server device, or according to any other suitable technique. For purposes of this discussion, the term "server device" may refer to an individual server device or a group of two or more server devices.

[0031] During operation, a mapping application 122 running within the client device 10 receives and transmits data to the server device 60. Thus, in one example, the client device 10 may be configured to navigate from an origin to a destination. InstructionsThe client device 10 may send a communication to the content-aware voice navigation generator 68 (implemented in the server device 60) requesting the content-aware voice navigation generator 68 to play the voice navigation instructions for the next maneuver. The content-aware voice navigation generator 68 may then generate a set of voice navigation instructions. Then, before playing voice navigation instructions for the next maneuver, the client device 10 may send characteristics of the media / audio content being played within the vicinity of the client device 10 to the content-aware voice navigation generator 68 and / or send an indication of the voice navigation instructions. The content-aware voice navigation generator 68 may determine how to adjust the voice navigation instructions based on the media / audio content and may send the adjusted voice navigation instructions or data for adjusting the voice navigation instructions to the client device 10. In other implementations, the client device 10 may send characteristics of the media / audio content being played within the vicinity of the client device 10 to the content-aware voice navigation generator 68 before the content-aware voice navigation generator 68 generates a set of voice navigation instructions. The content-aware voice navigation generator 68 may then generate a set of voice navigation instructions taking into account the characteristics of the media / audio content.

[0032] In some embodiments, the content-aware voice navigation generator 68 generates and provides each tailored voice navigation instruction to the client device 10 individually. In other embodiments, the content-aware voice navigation generator 68 generates and provides a set of text-based navigation instructions to the client device 10. Then, for each text-based navigation instruction, the content-aware voice navigation generator 68 receives characteristics of the media / audio content being played within the vicinity of the client device 10, generates and provides voice navigation instruction parameters for the text-based navigation instruction to the client device 10. As a result, the client device 10 generates and presents the voice navigation instructions according to the voice navigation instruction parameters. The voice navigation instruction parameters may include a language of the voice navigation instructions, a speaking rate of the voice navigation instructions, a timing of presenting the voice navigation instructions, etc.

[0033] Also, in some embodiments, the content-aware voice navigation generator 68 generates and provides to the client device 10 a set of text-based navigation instructions that include multiple versions of the same navigation instruction (e.g., a first version of the navigation instruction in English, a second version of the navigation instruction in Spanish, a third version of the navigation instruction that instructs the user to turn left in 1000 meters, a fourth version of the navigation instruction that instructs the user to turn left in 500 meters if the instruction is delayed until the end of a segment or sentence of the media content, etc.). For each navigation instruction, the content-aware voice navigation generator 68 then generates and provides to the client device 10 voice navigation instruction parameters for the navigation instruction. As a result, the client device 10 presents a version of the navigation instruction that corresponds to the voice navigation instruction parameters.

[0034] The content-aware voice navigation generator 68 may adjust voice navigation parameters for the plurality of navigation instructions based on the media content, such as adjusting the language for each of the remaining navigation instructions based on the language of the media content. In addition, the content-aware voice navigation generator 68 may adjust voice navigation parameters for each of the navigation instructions based on the media content, such as adjusting the timing of each of the navigation instructions to play the each of the navigation instructions after a particular sentence or segment within the media content.

[0035] The content-aware voice navigation generator 68 may adjust the voice navigation parameters based on a set of pre-stored rules. For example, the pre-stored rules may include a first rule that adjusts the language of the voice navigation instructions based on the language of at least one segment of the media content, at least a threshold number of segments of the media content, or at least a threshold amount of time of the media content. By adjusting the language of the voice navigation instructions to match the language of the media content, the user's comprehension of the voice navigation instructions is improved, thereby improving user safety. The pre-stored rules may also include a second rule that adjusts the speaking rate of the voice navigation instructions based on the speaking rate of at least one segment of the media content, at least a threshold number of segments of the media content, or at least a threshold amount of time of the media content. Again, by adjusting the speaking rate of the voice navigation instructions to more closely match the speaking rate of the media content, the user's comprehension and safety are improved.

[0036] Additionally, the pre-stored rules may include a third rule that adjusts the timing of the voice navigation instructions so that the voice navigation instructions are played after the end of a sentence or segment of the media content. By playing the voice navigation instructions after the end of a sentence or segment of the media content, the voice navigation instructions are more easily understood because there is no abrupt interruption in the content flow. This also improves user understanding and safety. In some implementations, the pre-stored rules may determine the level of complexity of the voice navigation instructions, the urgency level at which the voice navigation instructions must be played before arriving at a location for a maneuver, and the relevance level of the media content.

[0037] For example, the relevance level of the media content may be low if the media content is music and high if the media content is an audio tour of a city in which the user is driving or if the media content is an instructional video on how to perform a task the user plans to perform when they reach their destination. The pre-stored rules may compare the level of complexity of the voice navigation instructions, the urgency level that the voice navigation instructions must be played before arriving at a location for a maneuver, and / or the relevance level of the media content to determine when and how to interrupt the media content to play voice navigation instructions.

[0038] For example, when the urgency level at which voice navigation instructions must be played is high, the pre-stored rules may indicate to interrupt the media content even if the relevance level of the media content is high. This ensures that the urgent navigation instructions are still played as needed. On the other hand, when the urgency level at which voice navigation instructions must be played is medium or low and the relevance level of the media content is high, the pre-stored rules may indicate to wait to play the voice navigation instructions until the end of a sentence or segment in the media content, or to turn down the volume of the media content or to turn up the volume of the voice navigation instructions while playing the voice navigation instructions simultaneously with the media content. In another example, when the complexity level is high and the relevance level is high, the pre-stored rules may indicate to play the voice navigation instructions multiple times, once after the end of a first sentence or segment and a second time after the end of a second sentence or segment. Repeatedly playing complex instructions in certain instances to avoid unnaturally interrupting the media content improves user safety because the user is provided with instructions multiple times at clear points. In yet another example, if the complexity level exceeds the relevance level, the pre-stored rules may indicate to play voice navigation instructions immediately, whereas if the relevance level exceeds the complexity level, the pre-stored rules may indicate not to play voice navigation instructions or to wait until the end of the sentence or segment to play voice navigation instructions.

[0039] To play voice navigation instructions after the end of a sentence or segment of the media content, the client device 10 analyzes a current audio stream of the media content using audio feature classification and / or natural language processing models, as described in more detail below, to identify specific markers within the media content (e.g., end of a sentence, end of a segment, etc.). The client device 10 may continue to analyze the current audio stream in real-time or near real-time until the end of the sentence or segment is identified using the audio feature classification and / or natural language processing model. Then, once the end of the sentence or segment is identified, the client device 10 may play voice navigation instructions.

[0040] The voice navigation instructions may also be adjusted to account for the time delay in waiting until the end of a sentence or segment to play the voice navigation instructions. For example, initially the voice navigation instructions may be "100 meters, turn left." After the end of the sentence, the user may be 50 meters from the location for the maneuver, and therefore the voice navigation instructions may be adjusted to "50 meters, turn left." To adjust the voice navigation instructions to account for the time delay, the voice navigation instructions may initially be generated with a variable distance, such as "X meters, turn left." The mapping application 122 may determine the distance at the time the voice navigation instructions are presented as the distance between the user's current location and the location for the maneuver, and may replace "X" with the determined distance, or may round up or round the determined distance to the nearest 50 meters, 100 meters, 500 meters, etc. By doing this, the accuracy of the voice navigation instructions is not compromised by the delay in outputting the voice navigation instructions. In this manner, the methods described herein avoid unnaturally interrupting media content with the voice navigation instructions without compromising the accuracy of the voice navigation instructions. This improves understanding of voice navigation instructions and improves user safety.

[0041] In addition, the client device 10 may be configured to receive and / or view electronic media content using one or more voice navigational interfaces. Instructions In accordance with the present invention, the client device 10 may adjust the way in which the media content is presented according to the media content. More specifically, the client device 10 may, for example, pause the media content or adjust the volume of the media content (e.g., by communicating with the application via an API) if the media content is being played from an application running on the client device 10. In another example, if the media content is being played from another device communicatively coupled to the client device 10 (e.g., via a short-range communications link), the client device 10 may send a request to the other device to pause the media content being played on the other device or adjust the volume of the media content. This may cause the other device to pause the media content or adjust the volume of the media content. Pausing the media content advantageously avoids both the media content and the voice navigation instructions being played simultaneously, thereby ensuring that the clarity of the voice navigation instructions is not affected by the media content. Similarly, for example, lowering the volume of the media content helps the user hear and understand the voice navigation instructions. Thus, both of these concepts improve user safety.

[0042] Still further, the pre-stored rules may include a fourth rule that identifies a POI contained within the media content or other geographic topic discussed in the media content (e.g., Ancient Rome) and determines whether the POI is within a threshold radius of a waypoint along the route. If the POI is within a threshold radius of a waypoint along the route, the pre-stored rule may indicate to play information about the POI, such as "If you're interested, on the left is a Roman archaeological site." Additionally or alternatively, the pre-stored rule may indicate to play information about the POI, such as "Joe's Coffee is about a mile down the road on the left. Navigate there." Instructions The navigation system may then provide the user with suggestions to navigate to the POI, such as, "Joe's Coffee is about a mile down to the left. Would you like navigation directions there?". If the user then selects Instructions If the user indicates a desire to navigate to Joe's Coffee, the content-aware voice navigation generator 68 generates a voice navigation command from the user's current location to Joe's Coffee. Instructions A set of

[0043] Further, the pre-stored rules may include a fifth rule that determines the voice characteristics of a speaker of the media content, for example, when the media content is an audiobook or a podcast. The pre-stored rules may then identify a voice for playing voice navigation instructions that is distinct from the speaker of the media content, so that the user is alerted when the voice navigation instructions are presented. The pre-stored rules may select a voice recording from a set of pre-defined voice recordings by comparing the characteristics of the speaker's voice (e.g., the pitch, tone, and frequency of the voice) with the characteristics of each of the voice recordings, and identify the voice recording that is least similar to the speaker's voice. By identifying a voice for playing voice navigation instructions that is distinct from the voice of the speaker of the media content, the voice navigation instructions are more easily understood by the user.

[0044] In addition to or as an alternative to adjusting the voice navigation parameters based on a set of pre-stored rules, the content-aware voice navigation generator 68 may generate a machine learning model for adjusting the voice navigation instructions for a particular media / audio content. To generate the machine learning model, the content-aware voice navigation generator 68 obtains training data including a set of voice navigation instructions previously provided to the user, characteristics of the media content being played when the voice navigation instructions were provided, indicators of adjustments made by the user to the voice navigation instructions, and / or self-reported indicators of the user's satisfaction with the voice navigation instructions.

[0045] For example, a user who selects an option to share location data and / or other user data may transmit the set of voice navigation instructions presented by each client device 10 along with characteristics of the media content that was being played when the voice navigation instructions were presented.

[0046] The characteristics of the media content may include the language of the media content, the speaking rate of the media content, the voice characteristics of the speaker of the media content, the transcription of the media content, the length of the media content, the title of the media content, the type of media content (e.g., podcast, audiobook, song, etc.), POIs or other geographic topics mentioned in the media content, etc. In some implementations, the client device 10 determines the characteristics of the media content by communicating with the source of the media content (e.g., via a short-range communications link or an API). In other implementations, the client device 10 determines the characteristics of the media content by analyzing surrounding audio fingerprints in an area.

[0047] For example, a user may select an option to enable the mapping application 122 to analyze ambient audio in an area if the ambient audio is identified as electronic media content. The client device 10, and more specifically the mapping application 122, may first identify that media content is being played from a source separate from the mapping application by communicating with other applications 132 running on the client device 10 or other computing devices 92 within communication range of the client device 10. The mapping application 122 may also identify that media content is being played by comparing an audio fingerprint of the given media / audio content to the ambient audio fingerprints in the surrounding area.

[0048] For example, the mapping application 122 may obtain a library of audio fingerprints corresponding to media / audio content from popular music, audio books, radio shows, podcasts, etc. The mapping application 122 may extract fingerprints from the ambient audio, identify features of the ambient audio fingerprints, and compare the features of the ambient audio fingerprints to features of the audio fingerprints from the given media / audio content, for example, using machine learning techniques. The machine learning techniques may include linear regression, polynomial regression, logistic regression, random forests, boosting, nearest neighbors, Bayesian networks, neural networks, support vector machines, or any other suitable machine learning techniques. For example, frequency, pitch, tone, amplitude, etc. may be stored as audio fingerprint features. Features may be identified for entire songs, podcasts, audio books, etc., or for individual segments of the media content (e.g., every 8-second segment). Each of these audio fingerprints for the given media / audio content may then be compared to the features of the ambient audio fingerprints.

[0049] In some embodiments, the audio fingerprint features of a given media / audio content may be compared to the features of the ambient audio fingerprint using a nearest neighbor algorithm. The nearest neighbor algorithm may identify the audio fingerprint features of the given media / audio content that are closest to the features of the ambient audio fingerprint. The mapping application 122 may then determine that the ambient audio includes electronic media content if the ambient audio fingerprint features match or have more than a threshold amount of similarity with the audio fingerprint features of one of a song, audio book, radio show, podcast, etc. in the given media / audio content. The mapping application 122 may also determine that the ambient audio is a particular song, audio book, radio show, podcast, etc. in the given media / audio content that matches or has more than a threshold amount of similarity with the ambient audio fingerprint.

[0050] The mapping application 122 may then identify characteristics of the ambient audio based on characteristics of matching songs, audiobooks, radio shows, podcasts, etc. from the given media / audio content. In other implementations, such as when the mapping application 122 does not use audio fingerprinting techniques to identify matching songs, audiobooks, radio shows, podcasts, etc. from the given media / audio content, the mapping application 122 may run content analysis models, including voice recognition, speaker identification, audio feature classification, and natural language processing models, on the audio stream of the media content. These models may include neural networks or any other suitable type of machine learning models that analyze characteristics of the audio stream and compare the characteristics to training data to identify speakers, terms mentioned in the audio stream such as POIs or other geographic topics, specific markers within the media content (e.g., end of sentences, end of segments, etc.), the language of the media content, the speaking rate of the media content, etc.

[0051] For example, the model may be trained using training data including audio streams from several speakers, and characteristics of the audio streams (e.g., pitch, tone, frequency, amplitude, etc.) are classified according to each speaker. The mapping application 122 may analyze the audio streams of the media content to identify characteristics of the audio streams and apply the characteristics to the model to distinguish a speaker from several speakers having similar characteristics.

[0052] In another example, the model may be trained using training data that includes audio streams in several languages. The mapping application 122 may analyze the audio streams of the media content to identify characteristics of the audio streams and apply the characteristics to the model to identify a language from several languages ​​that have similar characteristics.

[0053] In yet another example, the model may be trained using training data including audio streams in which particular terms, phrases, sentences, etc. are spoken and characteristics of the audio streams are classified according to the respective terms, phrases, sentences, etc. The mapping application 122 may analyze the audio streams of the media content to identify characteristics of the audio streams and apply the characteristics to the model to identify terms, phrases, sentences, etc. from several terms, phrases, sentences, etc. that have similar characteristics.

[0054] In any event, for each presented voice navigation instruction or set of voice navigation instructions, the content-aware voice navigation generator 68 may obtain characteristics of the media content that was played when the voice navigation instructions were presented. In addition, the content-aware voice navigation generator 68 may obtain indications of adjustments made by the user to the voice navigation instructions, such as changing the language, changing the speech rate, changing the volume, muting the voice navigation instructions, requesting that the voice navigation instructions be repeated, etc. Also, in some embodiments, the content-aware voice navigation generator 68 may obtain self-reported indications of user satisfaction with the voice navigation instructions as additional training data. For example, the mapping application 122 may include a user control for indicating dissatisfaction with the voice navigation instructions and / or a text field for the user to explain his or her dissatisfaction. In this manner, the machine learning model may generate voice navigation instructions that are more likely to be followed by the user and less likely to annoy or disturb the user.

[0055] The set of voice navigation instructions, the voice navigation instruction parameters, the media content characteristics, the user adjustments to the voice navigation instruction parameters, and / or an indication of whether the user was satisfied with the voice navigation instructions may be provided as training data for generating a machine learning model using machine learning techniques. In some embodiments, a separate machine learning model may be generated for each voice navigation instruction parameter. For example, one machine learning model may be generated to determine the language of the voice navigation instructions. Another machine learning model may be generated to determine the timing of providing the voice navigation instructions. Yet another machine learning model may be generated to determine the speaking rate of the voice navigation instructions.

[0056] 3 illustrates example data 300 that may be used to tailor voice navigation instructions for particular media / audio content. In some embodiments, the content-aware voice navigation generator 68 may apply the data 300 to a set of pre-stored rules to tailor the voice navigation instructions. The data 300 may include media / audio content characteristics 310 and voice navigation instruction parameters 320. The media / audio content characteristics 310 may include a transcription of the media content, a speaking rate of the media content, a language of the media content, POIs contained within the media content, a length of the media content, a relevance level of the media content, a title of the media content, a type of media content, a voice characteristic of a speaker of the media content, etc.

[0057] A transcription of the media content may include the text of the media content as well as indicators of when there are pauses in the media content, such as the end of a sentence, the end of a segment, the beginning of the next sentence, the beginning of the next segment, etc.

[0058] The relevance level of the media content may be a relevance score, such as from 1 to 100, or may be a category, such as "very low," "low," "medium," "high," "very high," or may be indicated in any other suitable manner. The relevance level of the media content may be based on the subject matter of the media content, the user's current location, and / or voice navigation. Instructions For example, the relevance level may be high if the media content is an audio tour of a city in which the user is driving, or if the media content is an instructional video on how to perform a task that the user may perform at the destination.

[0059] The types of media content may include radio programs, podcasts, audiobooks, music, advertisements, television programs, movies, videos, etc. The characteristics of the speaker's voice may include the pitch, tone, frequency, or any other suitable characteristics of the speaker's voice.

[0060] Exemplary media / audio content characteristics 310 may include a transcription of the media content, a speaking rate of the media content, a language of the media content, POIs contained within the media content, a length of the media content, a relevance level of the media content, a title of the media content, a type of media content, voice characteristics of a speaker of the media content, etc., which are just a few examples of media / audio content 310 for ease of explanation only. Any suitable media / audio content characteristics 310 may be used, and a subset of the described characteristics 310 may be used.

[0061] The voice navigation instruction parameters 320 may include a type of maneuver, a location of the maneuver, a level of complexity of the maneuver, a level of urgency for playing the voice navigation instructions, a language for the voice navigation instructions, a speaking rate for the voice navigation instructions, a voice for the voice navigation instructions, a timing of the voice navigation instructions, and / or a volume for the voice navigation instructions.

[0062] The complexity level of a maneuver may be a complexity score, such as 1 to 100, or may be a category, such as “very low”, “low”, “medium”, “high”, “very high”, or may be indicated in any other suitable manner. The complexity level of a maneuver may be determined based on a maneuver type, such as a four-way turn, a six-way turn, a roundabout, a U-turn, a highway merge, a highway exit, etc. The complexity level may also be determined based on the amount of time or distance between the next maneuver and the previous maneuver. A maneuver that occurs immediately after a previous maneuver may have a higher complexity level. Additionally, the complexity level may be determined based on the number of lanes the user needs to change to perform the maneuver. For example, the mapping application 122 may compare the user's initial lane after performing a previous maneuver to the final lane for the user to perform the next maneuver. The complexity level may increase as the number of lane changes to perform the maneuver increases.

[0063] The urgency level for playing voice navigation instructions may be an urgency score, such as 1 to 100, may be a category, such as “very low”, “low”, “medium”, “high”, “very high”, or may be indicated in any other suitable manner. The urgency level of a maneuver may be determined based on the amount of distance and / or time until the user has to perform the maneuver. For example, the urgency score may be inversely proportional to the amount of distance and / or time until the user has to perform the maneuver. For example, the urgency score may be higher if the maneuver is 200 meters away from the user's current location than if the maneuver is 1 mile away from the user's current location.

[0064] The timing of the voice navigation instructions may include timing to present the voice navigation instructions, such as when the user is halfway between the location of a previous maneuver and the location of a subsequent maneuver. Volume may be indicated in decibels (dB) or categorized as low volume (e.g., below a first threshold decibel amount), medium volume (e.g., between a first threshold decibel amount and a second threshold decibel amount higher than the first threshold decibel amount), high volume (e.g., above a second threshold decibel amount), etc.

[0065] Exemplary voice navigation instruction parameters 320 may include a type of maneuver, a location of the maneuver, a level of complexity of the maneuver, a level of urgency for playing voice navigation instructions, a language of the voice navigation instructions, a speech rate of the voice navigation instructions, a voice for the voice navigation instructions, a timing of the voice navigation instructions, a volume of the voice navigation instructions, etc., but these are only a few examples of voice navigation instruction parameters 320 for ease of explanation only. Any suitable voice navigation instruction parameters 320 may be used, and a subset of the described parameters 320 may be used.

[0066] The data 300 also includes indications of adjustments to the voice navigation instructions based on media content characteristics and / or voice navigation instruction parameters. The content-aware voice navigation generator 68 may determine the adjustments to the voice navigation instructions by applying pre-stored rules to the media content characteristics and / or voice navigation instruction parameters. In other scenarios, the adjustments may be manual adjustments from a user selecting a user control.

[0067] For example, in response to Joe's Coffee being included in the advertisement, the content-aware voice navigation generator 68 may determine the location of a Joe's Coffee store and navigate to that location. InstructionsThe content-aware voice navigation generator 68 may communicate with the map data server 50 to compare the location of a Joe's Coffee to the waypoints along the route. If Joe's Coffee is within a threshold radius of one of the waypoints along the route, the content-aware voice navigation generator 68 may generate a recommendation to the user to make a detour to Joe's Coffee. Instructions . In another example, in response to the podcast speaking rate being 1.5x, the content-aware voice navigation generator 68 may increase the speaking rate of the voice navigation instructions to 1.5x and adjust the timing of the next voice navigation instruction to play after the end of a sentence in the podcast. In yet another example, in response to the media content including Spanish music, the content-aware voice navigation generator 68 may change the language of the voice navigation instructions to Spanish. In another example, in response to the podcast speaking rate being 2x in Hebrew, the content-aware voice navigation generator 68 may increase the speaking rate of the voice navigation instructions to 2x and change the language of the voice navigation instructions to Hebrew.

[0068] In other embodiments, the data 300 may be training data used to generate a machine learning model. In some embodiments, the training data 300 may be stored in the database 80. In addition to the media / audio content characteristics 310 and the voice navigation instruction parameters 320, the training data 300 may include data indicative of a driver's response to the voice navigation instructions. The data indicative of the driver's response to the voice navigation instructions may include adjustments made by the user to the voice navigation instructions, such as changing the language, changing the speech rate, changing the volume, muting the voice navigation instructions, requesting that the voice navigation instructions be repeated, etc. The data indicative of the driver's response to the voice navigation instructions may include self-reported indicators of user satisfaction with the voice navigation instructions as additional training data.

[0069] To generate the machine learning model, the content-aware voice navigation generator 68 may classify subsets of the training data 300 as corresponding to various adjustments to the voice navigation instructions. Some subsets may be classified in multiple adjustment categories, such as the last entry of the training data set 300 that includes adjustments to the speaking rate and language of the voice navigation instructions.

[0070] The content-aware voice navigation generator 68 may then analyze the subset to generate a machine learning model. The machine learning model may be generated using various machine learning techniques, such as regression analysis (e.g., logistic regression, linear regression, or polynomial regression), k-nearest neighbors, decision trees, random forests, boosting, neural networks, support vector machines, deep learning, reinforcement learning, Bayesian networks, etc. In some embodiments, the content-aware voice navigation generator 68 may generate a first machine learning model for determining a language of the voice navigation instructions. The content-aware voice navigation generator 68 may generate a second machine learning model for determining a speaking rate of the voice navigation instructions. The content-aware voice navigation generator 68 may generate a third machine learning model for determining a timing to provide the voice navigation instructions, a fourth machine learning model for identifying an audio recording for the voice navigation instructions, and a fifth machine learning model for determining a POI to reference in the voice navigation instructions.

[0071] For example, a machine learning model for determining when to provide voice navigation instructions may be a decision tree having a number of nodes connected by branches, where each node represents a test on media / audio content characteristics and / or voice navigation instruction parameters (e.g., is the media content highly relevant?), each branch represents an outcome of the test (e.g., yes), and each leaf represents an adjustment to the timing of the voice navigation instructions (e.g., play immediately, play after the end of a sentence in the media content, play after the end of a segment of the media content, etc.).

[0072] More specifically, the content-aware voice navigation generator 68 may generate a decision tree in which a first node corresponds to whether the urgency level of the voice instructions is high. If the urgency level is not high, the first branch may connect to a second node corresponding to whether the media content has more than 10 seconds remaining. If the second content has more than 10 seconds remaining, the second branch may connect to a third node corresponding to whether the relevance level is high. If the relevance level is high, the third branch may connect to a leaf node that may indicate that the voice navigation instructions should be presented after the end of a sentence in the media content. Although the decision tree includes one leaf node and three branches, this is merely an example for ease of explanation. Each decision tree may include any number of nodes, branches, and leaves with any suitable number and / or type of tests on media / audio content characteristics and / or voice navigation instruction parameters.

[0073] In any event, the content-aware voice navigation generator 68 may use pre-stored rules or machine learning techniques, as described above, to identify adjustments to the voice navigation instructions for the particular media / audio content. In some cases, the content-aware voice navigation generator 68 may then generate adjusted voice navigation instructions. As described, such modifications improve the clarity of the voice navigation instructions by avoiding conflicts or otherwise distinguishing them from other media content. For example, if the content-aware voice navigation generator 68 determines to change the language from English to Spanish, the content-aware voice navigation generator 68 may generate voice navigation instructions in Spanish from the starting location to the destination. Instructions Get a set of Spanish voice navigation Instructions to the client device 10. In another example, if the content-aware voice navigation generator 68 adds an additional recommendation to look right to see a particular monument mentioned in the media content, the content-aware voice navigation generator 68 may provide the additional recommendation to the client device 10.

[0074] In another example, the content-aware voice navigation generator 68 may provide instructions to the client device 10 to adjust parameters of the voice navigation instructions. For example, if the content-aware voice navigation generator 68 determines to change the speaking rate to 1.5 times, the content-aware voice navigation generator 68 may send instructions to the client device 10 to change the speaking rate to 1.5 times, and the client device 10 may play the voice navigation instructions at 1.5 times. In another example, if the content-aware voice navigation generator 68 determines to play voice navigation instructions at a particular point in time, such as the end of a sentence or segment in the media content, the content-aware voice navigation generator 68 may send instructions to the client device 10 to wait to play the voice navigation instructions until the end of the sentence or segment in the media content. The client device 10 may then analyze the audio stream of the media content to determine when the sentence or segment ends and then play the voice navigation instructions. As mentioned above, the content-aware voice navigation generator 68 can be implemented in the server device 60 or the client device 10. In a scenario in which the content-aware voice navigation generator 68 is implemented on the client device 10, the content-aware voice navigation generator 68 may provide instructions to a text-to-speech (TTS) engine running on the client device 10 to adjust parameters of the voice navigation instructions.

[0075] Exemplary Method for Presenting Content-Aware Voice Navigation Instructions - Patent application 4 shows a flow diagram of an example method 400 for generating content-aware voice navigation instructions. The method may be implemented in a set of instructions stored on a computer-readable memory and executable on one or more processors of the client device 10. For example, at least some of the steps of the method may be implemented by the mapping application 122.

[0076] At block 402, a voice navigation system is provided for navigating a user from a starting location to a destination along a route. Instructions A set of the map data is obtained via the mapping application 122. For example, the user may navigate from the user's current location to a destination via the mapping application 122. Instructions , and the client device 10 may forward the request to the navigation server 34. The navigation server 34 may then generate a set of voice navigation instructions and send the set of voice navigation instructions to the client device 10 for playback to the user.

[0077] Then, at block 404, electronic media content being played from a source other than the mapping application is identified. The source may be another application 132 running on the client device 10 or another computing device 92 in the vicinity of the client device 10. To identify the electronic media content, the mapping application 122 may communicate with the other application 132 running on the client device (e.g., via an API) or with other computing devices 92 in the vicinity of the client device 10, such as a vehicle head unit 14 or other client devices (e.g., via a short-range communication link). The communication may include an indication of the electronic media content being played by the other application / device 92, 132 and may include characteristics of the electronic media content, such as the language of the electronic media content, the speaking rate of the electronic media content, a transcription of the electronic media content, voice characteristics of the speaker of the electronic media content, etc. (block 406).

[0078] In addition to communicating with other applications / devices to identify media content, the mapping application 122 may compare ambient audio playing in an area that may have been captured by the microphone of the client device 10 to a set of audio fingerprints of popular media content, such as popular songs, radio shows, podcasts, etc., to identify the electronic media content. Once the electronic media content is identified, the mapping application 122 may identify characteristics of the electronic media content by running content analysis models, including speech recognition, speaker identification, audio feature classification, and natural language processing models, on the audio stream of the electronic media content (block 406). These models may include neural networks or any other suitable type of machine learning models that analyze characteristics of the audio stream and compare the characteristics to training data to identify the speaker, terms mentioned in the audio stream such as POIs, specific markers in the media content (e.g., end of sentence, end of segment, etc.), the language of the media content, the speaking rate of the media content, etc.

[0079] In any case, next, voice navigation Instructions The set of preferences may be tailored according to characteristics of the electronic media content (block 408). More specifically, in some implementations, Instructions The set of may be adjusted using pre-stored rules. For example, the pre-stored rules may include a first rule that adjusts the language of the voice navigation instructions based on a language of at least one segment of the media content, at least a threshold number of segments of the media content, or at least a threshold amount of time of the media content. The pre-stored rules may also include a second rule that adjusts the speech rate of the voice navigation instructions based on a speech rate of at least one segment of the media content, at least a threshold number of segments of the media content, or at least a threshold amount of time of the media content.

[0080] Additionally, the pre-stored rules may include a third rule that adjusts the timing of the voice navigation instructions so that the voice navigation instructions are played after the end of a sentence or segment of the media content. Still further, the pre-stored rules may include a fourth rule that identifies a POI contained within the media content and determines whether the POI is within a threshold radius of a waypoint along the route. If the POI is within a threshold radius of a waypoint along the route, the pre-stored rule may indicate to play information about the POI, such as "If you're interested, on the left is a Roman archaeological site." Additionally or alternatively, the pre-stored rule may indicate to play information about the POI, such as "Joe's Coffee is about a mile down the road on the left. Navigate there." Instructions The navigation path may indicate that the navigation system should provide the user with suggestions to navigate to the POI, such as "Joe's Coffee is about a mile down to the left. Would you like navigation directions there?"

[0081] Additionally, the pre-stored rules may include a fifth rule that determines a voice characteristic of a speaker of the media content, for example, if the media content is an audiobook or podcast. The pre-stored rules may then identify a voice for playing voice navigation instructions that is distinct from the speaker of the media content, such that the user is alerted when the voice navigation instructions are presented.

[0082] In addition to or as an alternative to adjusting voice navigation parameters based on a set of pre-stored rules, the mapping application 122 may generate a machine learning model for adjusting voice navigation instructions for particular media / audio content. The mapping application 122 may then apply characteristics of the electronic media content and / or parameters of the voice navigation instructions to the machine learning model to identify adjustments to the voice navigation instructions.

[0083] The mapping application 122 may then generate tailored voice navigation instructions. For example, if the mapping application 122 determines to change the language from English to Spanish, the mapping application 122 may generate voice navigation instructions in Spanish from the starting point to the destination. Instructions In another example, the mapping application 122 may provide instructions to the TTS engine to adjust parameters of the voice navigation instructions.

[0084] In another embodiment, voice navigation is performed according to the characteristics of the electronic media content. Instructions To adjust the set of Instructions The server device 60, and more specifically, the content-aware voice navigation generator 68, may transmit the set of voice navigation instructions, characteristics of the electronic media content, and / or voice navigation instruction parameters to the server device 60. Instructions The content-aware voice navigation generator 68 may obtain a set of pre-stored rules for adjusting the set of rules or generate a machine learning model for adjusting the voice navigation instructions for the particular media / audio content. The content-aware voice navigation generator 68 may apply the characteristics of the electronic media content and / or the voice navigation instruction parameters to the pre-stored set of rules or machine learning model to identify adjustments to the voice navigation instructions. The server device 60 then generates the adjusted voice navigation instructions. Instructions, or voice navigation Instructions 1 to the client device 10. Beneficially, by using the server device 60 instead of the client device 10 to identify adjustments to the voice navigation instructions, resources of the client device 10 are saved since the processing, time, and power consumption to perform this task are that of the server device 60. In addition, since the processing capabilities of the server device 60 may exceed those of the client device 60, moving this step to the server device 60 allows processing to be performed more efficiently, particularly in real-time scenarios, thereby allowing adjustments to be identified more quickly.

[0085] Then, in block 410, the adjusted voice navigation is Instructions is presented to the user via a speaker, for example.

[0086] Additional Considerations The following additional considerations apply to the foregoing discussion: Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed simultaneously, and the operations need not be performed in the order illustrated. Structures and functions presented as separate components in an exemplary configuration may be implemented as combined structures or components. Similarly, structures and functions presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter of this disclosure.

[0087] In addition, certain embodiments are described herein as including logic, or some components, modules, or mechanisms. A module may constitute either a software module (e.g., code stored on a machine-readable medium) or a hardware module. A hardware module is a tangible unit that can perform certain operations and may be configured or arranged in a certain manner. In an exemplary embodiment, one or more computer systems (e.g., standalone, client, or server computer systems), or one or more hardware modules of a computer system (e.g., a processor or group of processors), may be configured as a hardware module that operates by software (e.g., an application or application portion) to perform certain operations described herein.

[0088] In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a dedicated processor, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC)) to perform certain operations. A hardware module may also comprise programmable logic or circuitry that is temporarily configured by software to perform certain operations (e.g., as contained within a general-purpose processor or other programmable processor). It will be appreciated that the decision to implement a hardware module mechanically, in dedicated permanently configured circuitry, or in temporarily configured (e.g., configured by software) circuitry may be determined by cost and time considerations.

[0089] Thus, the term hardware should be understood to encompass tangible entities that are physically configured, permanently configured (e.g., wired), or temporarily configured (e.g., programmed) to operate in a particular manner or to perform particular operations described herein. As used herein, a "hardware-implemented module" refers to a hardware module. Given an embodiment in which the hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one time. For example, if the hardware modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as different hardware modules at different times. Thus, the software may, for example, configure the processor to configure a particular hardware module at one time and a different hardware module at a different time.

[0090] Hardware modules can provide information to and receive information from other hardware modules. Thus, the described hardware modules can be considered to be communicatively coupled. When multiple such hardware modules are present simultaneously, communication can be achieved through signal transmission (e.g., via appropriate circuits and buses) connecting the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, through the storage and retrieval of information in a memory structure that can be accessed by the multiple hardware modules. For example, one hardware module can perform an operation and store the output of that operation in a communicatively coupled memory device. An additional hardware module can then access the memory device at a later time to retrieve and process the stored output. Hardware modules can also initiate communication with input or output devices and operate on resources (e.g., collections of information).

[0091] The method 400 may include one or more functional blocks, modules, individual functions, or routines in the form of tangible computer-executable instructions stored in a non-transitory computer-readable storage medium and executed using a processor of a computing device (e.g., a server device, a personal computer, a smartphone, a tablet computer, a smartwatch, a mobile computing device, or other client computing device, as described herein). The method 400 may be included as part of any back-end server (e.g., a map data server, a navigation server, or any other type of server computing device, as described herein), e.g., as part of a client computing device module of the exemplary environment, or as part of a module external to such environment. Although the figure may be described with reference to other figures for ease of explanation, the method 400 may be utilized with other objects and user interfaces. Additionally, while the above description describes steps of the method 400 as being performed by a particular device (e.g., the server device 60 or the client device 10), this is done for purposes of explanation only. The blocks of the method 400 may be performed by one or more devices or other parts of the environment.

[0092] Various operations of the example methods described herein may be performed at least in part by one or more processors that are temporarily or permanently configured (e.g., by software) to perform the associated operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. Modules referred to herein may comprise processor-implemented modules in some example embodiments.

[0093] Similarly, the methods and routines described herein may be at least partially implemented in a processor. For example, at least some of the operations of the method may be performed by one or more processors or processor-implemented hardware modules. The execution of certain operations may not only reside in one machine, but may be distributed among one or more processors that are spread across several machines. In some exemplary embodiments, the processor may be located in a single location (e.g., in a residential environment, in an office environment, or as a server farm), while in other embodiments, the processor may be distributed across several locations.

[0094] The one or more processors may also operate to support execution of related operations in a "cloud computing" environment or as SaaS. For example, as indicated above, at least some of the operations may be performed by a group of computers (as examples of machines that include a processor), which operations are accessible via a network (e.g., the Internet) and one or more appropriate interfaces (e.g., APIs).

[0095] Moreover, the figures depict several embodiments of example environments for purposes of illustration only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be used without departing from the principles described herein.

[0096] Upon reading this disclosure, those skilled in the art will appreciate still further alternative structural and functional designs for providing content-aware voice navigation instructions through the principles disclosed herein. Thus, while specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise structures and components disclosed herein. Various modifications, changes and variations apparent to those skilled in the art may be made in the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the spirit and scope as defined in the appended claims. [Explanation of symbols]

[0097] 1 Environment 10 Mobile devices, client devices 12 Vehicles 14 Head unit, vehicle head unit, I / O module 16 Communication Links 18 Display 19 Sensors 20 Hardware Input Control 22 Hardware Input Control 24 Microphones 26 Speaker 30 Communication Network 32 User Interface (UI) 34 Navigation data server, navigation server 50 Map Data Server 60 Server Devices 62 processors 64 Memory 68 Content-Aware Voice Navigation Generator 80 Databases 92 Other Computing Devices 100 Communication Systems 114 I / O Module 116 Processor (CPU) 120 Memory 122 Mapping Applications, Geography Applications, Applications 126 Operating System (OS), OS 132 Client Applications, Applications 300 data, training data, training dataset 310 Media / Audio Content Characteristics, Characteristics 320 Voice navigation instruction parameters, parameters

Claims

1. 1. A method for generating content-aware navigation instructions, the method comprising: obtaining, by one or more processors in the client device via a mapping application, one or more voice navigation instructions for traveling from the origin to the destination along a route; identifying, by the one or more processors, electronic media content being played from a source other than the mapping application, the source executing at or proximate to the client device; determining, by the one or more processors, a characteristic of the electronic media content; determining characteristics of the electronic media content including at least one of a speaking rate of the electronic media content, a transcription of the electronic media content, a length of the electronic media content, or points of interest (POI) or geographic topics contained within the electronic media content; adjusting, by the one or more processors, at least one of the one or more voice navigation instructions according to the characteristic of the electronic media content; presenting, by the one or more processors, the at least one tailored voice navigation instruction to a user; Including, method.

2. adjusting at least one of the one or more voice navigation instructions; adjusting a timing at which at least one of the one or more voice navigation instructions is presented; adjusting a language in which at least one of the one or more voice navigation instructions is presented; adjusting the rate at which at least one of the one or more voice navigation instructions is presented; or Providing recommendations regarding points of interest (POIs) along the route. At least one of The method of claim 1.

3. determining a characteristic of the electronic media content, determining a rate at which the electronic media content is presented by the source; determining a language in which the electronic media content is presented by the source; identifying a pause in the electronic media content; or identifying points of interest (POIs) or geographic topics contained within said electronic media content; At least one of The method according to claim 1 or 2.

4. adjusting at least one of the one or more voice navigation instructions according to the characteristic of the electronic media content; determining, by the one or more processors, a level of maneuver complexity in the at least one voice navigation instruction, the level of maneuver complexity being determined as a function of a maneuver type indicated by the voice navigation instruction; determining, by the one or more processors, a relevance level of the electronic media content, the relevance level being determined based on a subject matter of the electronic media content, a current location of the user, and / or a destination of the voice navigation instructions; comparing, by the one or more processors, the level of maneuver complexity of at least one voice navigation instruction to the relevance level of the electronic media content; determining, by the one or more processors, whether to present the at least one voice navigation instruction based on the comparison; Including, 4. The method according to any one of claims 1 to 3.

5. determining whether to present the at least one voice navigation instruction based on the comparison, presenting, by the one or more processors, the at least one voice navigation instruction in response to determining that the level of maneuver complexity of the at least one voice navigation instruction exceeds the relevance level of the electronic media content. The method of claim 4.

6. determining whether to present the at least one voice navigation instruction based on the comparison, in response to determining that the relevance level of the electronic media content exceeds the maneuver complexity level of the voice navigation instructions, not presenting, by the one or more processors, the at least one voice navigation instruction. The method of claim 4.

7. identifying electronic media content being played from a source different from the mapping application, obtaining, by the one or more processors, audio playback data from an audio application running on the client device distinct from the mapping application; obtaining, by the one or more processors, audio playback data from a device communicatively coupled to the client device; or comparing, by the one or more processors, the ambient audio fingerprint to one or more audio fingerprints of the given media content. At least one of 7. The method according to any one of claims 1 to 6.

8. adjusting at least one of the one or more voice navigation instructions; identifying, by the one or more processors, ends of sentences within the electronic media content; presenting, by the one or more processors, at least one of the one or more voice navigation instructions after completion of the sentence; Including, 8. The method according to any one of claims 1 to 7.

9. adjusting, by the one or more processors, how the electronic media content is presented in accordance with the one or more voice navigation instructions.

9. The method of claim 1, further comprising:

10. adjusting the at least one voice navigation instruction according to the characteristic of the electronic media content; adjusting the at least one voice navigation instruction by applying the characteristics of the electronic media content to a set of pre-stored rules; or generating a machine learning model for tailoring voice navigation instructions to particular electronic media content; and tailoring the at least one voice navigation instruction by applying the characteristics of the electronic media content to the machine learning model. At least one of 10. The method according to any one of claims 1 to 9.

11. 1. A client device for generating content-aware navigation instructions, the client device comprising: A speaker and one or more processors; a processor coupled to the one or more processors and the speaker, the processor coupled to the one or more processors ... and the speaker coupled to the one or more processors, the processor coupled to the one or more processors, obtaining, via a mapping application, one or more voice navigation instructions for traveling along a route from an origin to a destination; an act of identifying electronic media content being played from a source different from the mapping application, the source running at or proximate to the client device; determining a characteristic of the electronic media content, determining that the characteristics of the electronic media content include at least one of a speaking rate of the electronic media content, a transcription of the electronic media content, a length of the electronic media content, or points of interest (POI) or geographic topics contained within the electronic media content; adjusting at least one of the one or more voice navigation instructions according to the characteristic of the electronic media content; presenting the at least one tailored voice navigation instruction to a user via the speaker; and and a computer readable memory storing instructions for causing the Equipped with Client device.

12. To adjust at least one of the one or more voice navigation instructions, the instructions may include providing to the client device: adjusting a timing at which at least one of the one or more voice navigation instructions is presented; adjusting a language in which at least one of the one or more voice navigation instructions is presented; adjusting a rate at which at least one of the one or more voice navigation instructions is presented; or Providing recommendations regarding points of interest (POIs) along said route. Execute at least one of the following: The client device of claim 11.

13. To determine a characteristic of the electronic media content, the instructions may include: determining a rate at which the electronic media content is presented by the source; determining a language in which the electronic media content is presented by the source; identifying a pause in the electronic media content; or identifying points of interest (POIs) or geographic topics contained within said electronic media content; Execute at least one of the following: A client device according to claim 11 or 12.

14. to adjust at least one voice navigation instruction according to the characteristic of the electronic media content, the instruction being sent to the client device; an operation of determining a level of complexity of a maneuver in the at least one voice navigation instruction, the level of complexity of the maneuver being determined in response to a maneuver type indicated by the voice navigation instruction; an act of determining a relevance level of the electronic media content, the relevance level being determined based on a subject matter of the electronic media content, a current location of the user, and / or a destination of the voice navigation instructions; comparing the maneuver complexity level of the at least one voice navigation instruction to the relevance level of the electronic media content; determining whether to present the at least one voice navigation instruction based on the comparison; and Execute the A client device according to any one of claims 11 to 13.

15. to the client device to determine whether to present the at least one voice navigation instruction based on the comparison; in response to determining that the level of maneuver complexity of the at least one voice navigation instruction exceeds the relevance level of the electronic media content, causing presentation of the at least one voice navigation instruction.

15. The client device of claim 14.

16. To identify electronic media content being played from a source other than the mapping application, the instructions include: obtaining audio playback data from an audio application running on the client device that is different from the mapping application; obtaining audio playback data from a device communicatively coupled to the client device; or Comparing the ambient audio fingerprint to one or more audio fingerprints of the given media content. Execute at least one of the following: A client device according to any one of claims 11 to 15.

17. To adjust at least one of the one or more voice navigation instructions, the instructions may include providing to the client device: identifying an end of a sentence within the electronic media content; presenting at least one of the one or more voice navigation instructions after the end of the sentence. Execute the A client device according to any one of claims 11 to 16.

18. The instructions may further include, on the client device: adjusting the manner in which the electronic media content is presented in accordance with the one or more voice navigation instructions. A client device according to any one of claims 11 to 17.

19. When executed by one or more processors, the one or more processors are obtaining, via a mapping application, one or more voice navigation instructions for traveling along a route from an origin to a destination; an act of identifying electronic media content being played from a source different from the mapping application, the source running at or proximate to the client device; acts of determining characteristics of the electronic media content, the characteristics of the electronic media content including at least one of a speaking rate of the electronic media content, a transcription of the electronic media content, a length of the electronic media content, or points of interest (POI) or geographic topics contained within the electronic media content; adjusting at least one of the one or more voice navigation instructions according to the characteristic of the electronic media content; presenting the at least one tailored voice navigation instruction to a user via a speaker; and A computer-readable memory storing instructions for executing the

20. To adjust at least one of the one or more voice navigation instructions, the instructions may include: adjusting a timing at which at least one of the one or more voice navigation instructions is presented; adjusting a language in which at least one of the one or more voice navigation instructions is presented; adjusting a rate at which at least one of the one or more voice navigation instructions is presented; or Providing recommendations regarding points of interest (POIs) along said route. Execute at least one of the following:

20. The computer readable memory of claim 19.

Citation Information

Patent Citations

  • Broadcast receiving device

    JP2010117476A

  • Navigation device

    JP2011095142A

  • Device, method, and program for outputting voice

    JP2019117324A