SONIFICATION OF NAVIGATION SEARCH RESULTS

Abstract

A method for sonifying search results, performed while a user is in the passenger compartment of a vehicle, comprises: receiving a response to a user-submitted query, the response containing a list of potential matches to the query, each potential match containing corresponding embedded metadata. For each potential match, the method further comprises: extracting the embedded metadata corresponding to the potential match; determining, based on the embedded metadata, a spatially located position within a playback sound field that the user can perceive as the sound source of the potential match; and outputting audio signals characterizing the potential match through a loudspeaker array in the passenger compartment to generate the playback sound field.The user perceives the potential hit as originating from the sound source at the spatially arranged point within the playback sound field.

Description

INTRODUCTION The information in this section serves to present the general context of the disclosure. Works of the inventors mentioned herein, insofar as they are described in this section, as well as aspects of the description that may not have been prior art at the time of filing, are neither expressly nor implicitly admitted as prior art against the present disclosure. The present disclosure relates generally to the sonification, or sonic representation, of navigation search results. Navigation systems in vehicles have become an indispensable aid for motorists, providing real-time directions and information about points of interest. These systems typically present navigation search results through visual displays that include a map and relevant data such as the names and addresses of destinations. The visual interface often includes icons or markers indicating the locations of the search results, and users can interact with the display to select these results or obtain further information about them. Naturally, these navigation systems typically require the user to look at the display to fully understand the location of each search result. This visual reliance means the user must interpret the information shown on the screen to understand the arrangement and proximity of various destinations to the vehicle's current location. While some systems offer auditory feedback, such as spoken directions, to assist the driver in selecting a point of interest from a list of search results, conveying all location information via verbal prompts can be time-consuming. Therefore, spatial cues regarding the direction and distance of a point of interest can be understood more quickly and easily by the vehicle user. US 2014 / 0019037 A1 discloses a navigation system comprising a hearing aid configured to be worn on the head and equipped with loudspeakers to emit sounds toward the user's ears. The navigation system further comprises a GPS unit for determining the system's geographic position, a sound generator connected to output audio signals to the loudspeakers, and a processor configured to select points of interest in the system's environment and to control the sound generator to output audio signals that sequentially represent spoken information about the selected points of interest. SUMMARY One aspect of the disclosure provides a computer-implemented method for sonifying navigation search results which, when executed on data processing hardware, causes the data processing hardware to perform operations that include: while a user is in the passenger compartment of a vehicle: receiving audio data characterizing a spoken utterance of the user's query, recorded by a microphone in the passenger compartment of the vehicle, wherein the audio data characterizing the spoken utterance of the query indicates a specific location of the user within the passenger compartment of the vehicle; receiving a response to a user-made query, wherein the response contains a list of potential matches to the query, each potential match containing appropriate embedded metadata.For each potential hit, the operations also include: extracting the embedded metadata corresponding to the potential hit; determining, based on the embedded metadata, a spatially located position within a playback sound field that the user can perceive as the sound source of the potential hit; and outputting audio signals that characterize the potential hit through a loudspeaker array in the passenger compartment to generate the playback sound field, with the playback sound field centered on the user's respective location within the vehicle's passenger compartment. The user perceives the potential hit as originating from the sound source at the spatially located position within the playback sound field. Implementations of the disclosure may include one or more of the following optional features. In some implementations, the operations also include receiving vehicle location information. In these implementations, playback of the output audio signals may include: modulating the output audio signals based on the distance between the vehicle's location information and the potential match's embedded metadata. In some examples, the operations also include, while playing back the output audio signals that characterize the potential match: simultaneously displaying a corresponding visual cue in a graphical user interface of the vehicle. In some implementations, the operations also include: identifying a user preference associated with one of the potential matches in the list of potential matches, where the user preference is stored in a user profile. In these implementations, playing back the output audio signals that characterize the potential match with the user preference involves playing back a user-defined audio output signal. In some implementations, the playback sound field represents locations outside the passenger compartment. The details of one or more implementations of the disclosure are set forth in the accompanying drawings and in the description below. Further aspects, features, and advantages will become apparent from the description and the drawings, as well as from the claims. BRIEF DESCRIPTION OF THE DRAWINGS The drawings described here serve only to illustrate selected configurations and are not intended to limit the scope of this disclosure. Fig. 1 is a schematic representation of an exemplary system for the sonification or sonic representation of navigation search results. Fig. 2 is a schematic view of exemplary components of the system in Fig. 1. Fig. 3 is a schematic view of exemplary components of the system in Fig. 1. Fig. 4 is a schematic representation of a graphical user interface for the system in Fig. 1. Fig. 5 is a flowchart of an exemplary arrangement of operations for a method for the sonification or sonic representation of navigation search results. The corresponding reference numbers denote the corresponding parts in the drawings. DETAILED DESCRIPTION Exemplary configurations are now described in more detail with reference to the accompanying drawings. Exemplary configurations are provided so that this disclosure is thorough and conveys the full scope of the disclosure to those skilled in the art. Specific details are listed, such as examples of specific components, devices, and processes, to provide a thorough understanding of the configurations of this disclosure. It is clear to those skilled in the art that specific details need not be used, that exemplary configurations can be implemented in many different forms, and that the specific details and exemplary configurations should not be interpreted in such a way as to limit the scope of the disclosure. The terminology used here serves only to describe certain exemplary configurations and is not intended to be restrictive. As used here, the singular articles "a," "an," and "the" can also include the plural forms unless the context clearly indicates otherwise. The terms "comprises," "comprehensive," "containing," and "exhibiting" are inclusive and therefore specify the presence of features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The procedural steps, processes, and operations described here are not to be interpreted as necessarily being carried out in the order discussed or presented, unless they are explicitly identified as such.Additional or alternative steps can be applied. When an element or layer is described as "on," "engaging," "connected," "attached to," or "coupled" with another element or layer, it may be directly on, engaged, connected, attached, or coupled to that other element or layer, or there may be intervening elements or layers. Conversely, when an element is described as "directly on," "directly engaged with," "directly connected to," "directly attached to," or "directly coupled to" another element or layer, there must be no intervening elements or layers. Other words used to describe the relationship between elements should be interpreted similarly (e.g., "between" versus "directly between," "next to" versus "directly beside," etc.).As used here, the term “and / or” includes all combinations of one or more of the related listed elements. The terms "first," "second," "third," etc., may be used here to describe different elements, components, regions, layers, and / or sections. These elements, components, regions, layers, and / or sections should not be restricted by these terms. These terms may only be used to distinguish one element, component, region, layer, or section from another. Terms such as "first," "second," and other numerical terms do not imply any sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer, or section discussed below could be referred to as a second element, component, region, layer, or section without deviating from the lessons of exemplary configurations. In this application, including the definitions below, the term "module" may be replaced by the term "circuit". The term "module" may refer to, be part of, or include: an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; memory (shared, dedicated, or group) that stores the code executed by a processor; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, e.g., in a system-on-a-chip. The term "code," as used above, can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, and / or objects. The term "shared processor" refers to a single processor that executes some or all of the code from multiple modules. The term "group processor" refers to a processor that, in combination with other processors, executes some or all of the code from one or more modules. The term "shared memory" refers to a single memory that stores some or all of the code from multiple modules. The term "group memory" refers to a memory that, in combination with other memories, stores some or all of the code from one or more modules. The term "memory" can be a subset of the term "computer-readable medium."The term "computer-readable medium" excludes transitory electrical and electromagnetic signals propagating through a medium and can therefore be considered tangible and non-transient storage. Non-restrictive examples of non-transient storage include tangible, computer-readable media, including non-volatile memory, magnetic storage, and optical storage. The devices and methods described in this application can be implemented in whole or in part by one or more computer programs executed by one or more processors. The computer programs contain processor-executable instructions stored on at least one non-transitory, tangible, computer-readable medium. The computer programs may also contain and / or access stored data. A software application (i.e., a software resource) can refer to computer software that causes a computing device to perform a task. In some examples, a software application may be called an "application," "app," or "program." Examples of applications include system diagnostics applications, system administration applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications. Non-transitory memory can be physical devices used to temporarily or permanently store programs (e.g., instruction sequences) or data (e.g., program status information) for use by a computer. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware such as boot programs). Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and floppy disks or tapes. These computer programs (also referred to as programs, software, software applications, or code) contain machine instructions for a programmable processor and may be implemented in a procedural and / or object-oriented high-level language and / or in assembly / machine language. The terms "machine-readable medium" and "computer-readable medium" used here refer to any computer program product, non-transient computer-readable medium, apparatus, and / or device (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) that serves to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal.The term "machine-readable signal" refers to any signal that serves to provide machine instructions and / or data to a programmable processor. Various implementations of the systems and techniques described herein can be realized in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system comprising at least one programmable processor, which can be used for special or general purposes and is coupled such that it receives data and instructions from and transmits data and instructions to a storage system, as well as at least one input device and at least one output device. The processes and logical sequences described in this description can be executed by one or more programmable processors, also known as data processing hardware, which run one or more computer programs to perform functions by responding to input data and producing outputs. The processes and logical sequences can also be executed by specialized logic circuits, such as an FPGA (Field Programmable Gate Array) or an ASIC (application-specific integrated circuit). Processors suitable for executing a computer program include, for example, both general-purpose and specialized microprocessors, as well as one or more processors from any type of digital computer. Generally, a processor receives instructions and data from read-only memory, random-access memory, or both.The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or is operationally connected to them to receive data from or transmit data to them. However, a computer does not necessarily have to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable media; magneto-optical disks; and CD-ROM and DVD-ROM disks.The processor and memory can be supplemented by special logic circuits or integrated into them. To enable interaction with a user, one or more aspects of the revelation can be implemented on a computer that has a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display), or touchscreen monitor to show information to the user, and optionally a keyboard and pointing device, such as a mouse or trackball, with which the user can input information into the computer. Other types of devices can also be used for user interaction; feedback to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and user input can be received in any form, including auditory, verbal, or tactile input.Furthermore, a computer can interact with a user by sending and receiving documents to and from a device used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser. As shown in Fig. 1, in some embodiments, a system 100 comprises a vehicle 10 that communicates with a remote system 60 via a network 70. The vehicle 10 and / or the remote system 60 comprise a request handler 40 and a sound representation or sonification system 50, respectively, which each run while a user 102 is in a passenger compartment 300 of the vehicle 10. In short, and as described in more detail below, the request handler 40 receives a request 24 from the user 102 and generates a response 202 to the request 24. The response 202 may, for example, contain a list of navigation search results related to the vehicle 10. The sonification system 50 then executes a sonification model 200 (Fig. 2) which is configured to receive the response 202 to the request 24 made by the user 102 and to output the response 202 into a playback sound field 302 (Fig.3) converts, which assigns the direction and distance of the response 202 to one or more spatially arranged locations 306 within the playback sound field 302. It should be noted that, instead of relying on verbal prompts to guide the user 102 through the response 202, which are based on higher-level cognitive mechanisms that the user 102 must follow, the playback sound field 302 conveys the response 202 by communicating with the user 102 through auditory spatial cues that are processed by lower-level perceptual mechanisms. In this way, the user 102 can understand the response 202 more quickly without having to consult a visual display (e.g., a graphical user interface 18) in the vehicle 10. In the example shown, the request handler 40 and the sonification system 50 are implemented in the vehicle 10. However, the request handler 40 and the sonification system 50 can also be implemented in any other propulsion system, e.g., in motorcycles, trucks, off-road vehicles, agricultural equipment, trains, aircraft, and the like. The vehicle 10 comprises data processing hardware 12 and storage hardware 14, in which instructions are stored that, when executed on the data processing hardware 12, cause the data processing hardware 12 to perform operations. While the request handler 40 and the sonification system 50 are described as being implemented by the data processing hardware 12 and the storage hardware 14 of the vehicle 10, their respective operations can also be implemented on other data processing devices (e.g., data processing devices that communicate with the vehicle 10), such as...a smartphone, a tablet, a smart display, a desktop / laptop, a smart watch or smartwatch, a smart device or smart glasses / headset. As shown in Fig. 1, the vehicle 10 further comprises a speaker array 16 (also referred to as a loudspeaker array 16), a graphical user interface 18, and a microphone array 20, each located in the passenger compartment 300 of the vehicle 10, through which the user 102 can interact with the request handler 40. The microphone array 20 can comprise one or more microphones 20 configured to capture acoustic sounds (i.e., audio data) that characterize utterances such as speech directed to the request handler 40. The loudspeaker array 16 can comprise two or more loudspeakers capable of outputting audio such as music and / or synthesized speech from the request handler 40.While the loudspeaker assembly 16 and the microphone assembly 20 are generally depicted as being located within a headliner of the interior of the vehicle 10, as if arranged at a front section of the vehicle 10, it should be clear that the loudspeaker assembly 16 and / or the microphone assembly 20 may be distributed throughout the entire passenger compartment 300 of the vehicle 10. The vehicle 10 may also include a sensor system 22 comprising a global positioning system (GPS), one or more cameras, a frontal collision avoidance system, a radar (radio detection and ranging), a lidar (light detection and ranging) capable of acquiring image data, and other external sensors of the vehicle 10. Although the sensor system 22 shown in Fig. 1 is located at the front of the vehicle 10, it should be noted that the sensor system 22 may include sensors located throughout the vehicle 10.The sensor system 22 can, for example, enable 360-degree environmental sensing of the area around the vehicle 10. Network 70 can include a wireless local area network (WLAN) that facilitates communication and interoperability between the vehicle 10 and the remote system 60 within the vehicle 10's vicinity. Network 70 can thus include Wireless Fidelity (WiFi®) (e.g., IEEE 802.11), Low-Rate Wireless Personal Area Networks (e.g., IEEE 802.15.4), Worldwide Interoperability for Microwave Access (WiMAX), 3G, 4G, Long Term Evolution (LTE), 5G, Digital Subscriber Line (DSL), Bluetooth, Near Field Communication (NFC), or other wireless standards, or Ethernet (e.g., IEEE 802.3). System 100 can additionally include one or more access points (APs) (not shown) configured to enable wireless communication between the vehicle 10 and the remote system 60.The remote system 60 (e.g., server, cloud computing environment) also includes data processing hardware 62 and storage hardware 64, in which instructions are stored that, when executed on the data processing hardware 62, cause the data processing hardware 62 to perform operations. In some examples, the execution of the request handler 40 and the sonification system 50 is distributed between the vehicle 10 and the remote system 60. As shown in Fig. 1, the request handler 40 manages the requests of the user 102, who is located in the passenger compartment 300 of the vehicle 10. The request handler 40 may, for example, include a speech recognizer (not shown) that employs an automatic speech recognition model capable of performing speech recognition or semantic interpretation of audio data corresponding to the request 24 made by the user 102. The request handler 40 may also include a natural language understanding (NLU) module.The system includes a Natural Language Understanding (NLA) component that performs a query interpretation of query 24 to identify voice commands within query 24 and retrieves a response 202 containing a list of potential matches 204 related to query 24. For example, user 102 might submit a query 24 "Find an EV charging station," whereupon query handler 40 generates a response 202 containing a list of potential matches 204 (i.e., EV charging stations near the location of vehicle 10) related to query 24. These potential matches 204 can also be referred to as points of interest (POIs) to which user 102 wishes to navigate in order to access services (e.g., charging electric vehicles). With continued reference to Fig. 1 and Fig. 2, the sonification system 50 executes the sonification model 200, which is configured to receive the response 202 generated by the request handler 40. The sonification model 200 comprises a signal generator 210 and a modulator 220. Additionally, the sonification model 200 has access to a data store 230 and is stored on the memory hardware 14, 64 of the system 100. As shown, the response 202 can contain, in addition to the list of potential matches 204, embedded metadata 206 for each potential match 204 in the list of potential matches 204. The embedded metadata 206 can, for example, include one or more of the following information: location information for the potential match 204, distances to the potential match 204, or capabilities (e.g., the charging speed of an EV charging station) of the potential match 204.The signal generator 210 can receive the list of potential matches 204 and their respective embedded metadata 206 and process the potential matches 204 and their respective embedded metadata 206 to identify the correct output audio signal 304 for the response 202. For each potential match 204 in the list of potential matches 204, the signal generator 210 can extract the embedded metadata 206 that corresponds to the potential match 204. The signal generator 210 can then access the data storage 230 based on the extracted embedded metadata 206 to obtain one or more output audio signals 304 that correspond to the content of the response 202.In other words, the output audio signals 304 in data storage 230 can be categorized depending on the category of the potential match 204, with different categories of potential matches 204 having different corresponding output audio signals 304. In this example, based on the respective extracted embedded metadata 206 of the response, the signal generator 210 can recognize that the potential matches 204 all relate to the category of cafes and retrieve output audio signals 304 from data storage 230 that correspond to a "dripping" sound. In other examples, where the signal generator 210 recognizes, based on the respective extracted embedded metadata 206 of the response, that the potential matches 204 all relate to the category of EV charging stations, the signal generator 210 retrieves output audio signals 304 from data storage 230 that correspond to a "humming" sound. It should be noted that the data store 230 can store both standard (i.e., created by a vehicle equipment manufacturer 10) output audio signals 304 and user-defined output audio signals 304. For example, the data store 230 can contain a user profile in which the user settings assigned to user 102 are stored. Here, user 102 can create their user profile with user preferences, including the user-defined output audio signals 304 recorded by user 102. In this case, the user settings can assign the user-defined output audio signal 304 either to a specific point of interest (i.e., a potential hit 204) or to a category of points of interest. The specific point of interest can include a potential hit 204 that user 102 has marked and / or frequently visited.For example, if a user 102 has repeatedly navigated to a particular grocery store, the user profile can store a personalized, custom output audio signal 304 as a user preference. This signal is unique to that specific grocery store and differs from the audio output signal 304 of other grocery stores. On subsequent requests, the signal generator 210 can detect when the grocery store in question is present in the list of potential matches 240 and play the output audio signal 304 associated with the user preference. By personalizing the output audio signal 304 based on the user's preferences, the user 102 receives a cue that distinguishes the grocery store in question from the list of potential matches 204 when the custom output audio signals 304 are played. Referring further to Fig. 2 and Fig. 3, the modulator 220 receives as input the extracted embedded metadata 206 for each of the corresponding potential hits 204 and the corresponding output audio signals 304 for each of the potential hits 204. For each potential hit 204, and based on the extracted embedded metadata 206, it determines a spatially located position 306 within a playback sound field 302, which the user 102 can perceive as the sound source 308 of the potential hit 204. Here, the modulator 220 can also receive as input the location information 208 of the vehicle 10 and determine the spatially located position 306 within the playback sound field 302 based on the location information 208 of the vehicle 10 relative to the potential hit 204.In other words, the modulator 220 can determine a specific direction within the passenger compartment 300 as the sound source 308 of the potential hit 204, as perceived by the user. Subsequently, for each potential hit 204, the modulator 220 reproduces the output audio signal 304, which characterizes the potential hit 204, via the loudspeaker arrangement 16 in the passenger compartment 300 to generate the playback sound field 302. The user 102 perceives the potential hit 204 as originating from the sound source 308 at the spatially arranged location 306 within the playback sound field 302. In generating the playback sound field 302, the modulator 220 embeds the extracted embedded metadata 206 and / or the location information 208 into the output audio signals 304 themselves, for example by taking into account interaural time differences, interaural level differences, direct-to-reflected sound ratios, etc.applies to give the user 102 spatial clues. In some cases, the modulator 220 processes / modifies the output audio signal 304 before playing it back to generate the playback sound field 302. In these cases, the modulator can modulate the output audio signals 304 based on the distance between the location information 208 of vehicle 10 and the extracted embedded metadata 206 of the potential target 204. For example, the modulator 220 can take into account the distance between vehicle 10 and the potential target 204 and adjust the volume level of the output audio signal 304 to indicate whether the potential target 204 is near (i.e., by increasing the volume of the output audio signal 304) or far away (i.e., by decreasing the volume of the output audio signal 304). Similarly, the modulator 220 can modify the sound attributes (e.g., pitch, timbre, perceived size, duration, repetitions, repetition rate, etc.).) of the output audio signal 304 to communicate the capabilities (e.g. fast charging or standard EV charging stations) specified by the extracted embedded metadata 206 of the potential match 204. Figure 3 shows the passenger compartment 300 of the vehicle 10 with the user 102 inside. Here, the modulator 320 (via the loudspeaker arrangement 16) reproduces the output audio signals 304a-304d to generate the spatial playback sound field 302, which assigns the specific direction and / or distance for each potential hit 204 to a spatially arranged point 306a-d. The user 102 perceives a sound source 308a-d of each potential hit 204 as originating from the respective spatially arranged point 306a-d within the playback sound field 302. As shown, the playback sound field 302 fills the interior 300 of the vehicle 10. However, it should be clear that the playback sound field represents 302 locations outside the passenger compartment 300 of the vehicle 10 to which the user can navigate 102. The modulator 220 can play back the output audio signals 304a-304d individually in the same order in which the potential hits 204 appear in the list of potential hits 204. In other cases, the modulator 220 can order the playback of the output audio signals 304 based on the probability (i.e., based on the user 102's preferences) that the user 102 will select the potential hit 204 as a navigation target. Optionally, the modulator 220 can play back the output audio signals first in a first round and then in a second round that includes additional auditory information (e.g., distance, orientation, capabilities) about the potential hit 204. As shown in Fig. 4, the sonification system 50 also integrates visual information onto a screen 400 of a graphical user interface 18 of the vehicle 10 in some cases. As shown, the screen 400 displays a map application with a response 202 containing two potential matches 204a, 204b to the query "Find an EV charging station". Here, the graphical user interface 18 can also include visual cues (i.e., graphical elements) 402a, 402b, which are displayed on the screen 400 of the graphical user interface 18. During the playback of the output audio signals 304, which identify the list of potential matches 204a, 204b, the sonification system 50 can simultaneously display the visual cues 402a, 402b. Here, the sonification system 50 can highlight any visual cue 402a, 402b (i.e., by changing the color, size, emphasis, shape, etc.).) when the output audio signal 304 is played back for the corresponding potential hit 204a, 204b. It should be noted that the playback sound field 302 can be optimized for more than one user position (e.g., driver, front passenger, rear passenger) in the vehicle 10. In other words, the playback sound field 302 can be rendered or played back based on the location of user 102 within the vehicle 10. Since user 102, who made the request 24, is likely the person controlling the navigation of the vehicle 10, the playback sound field 302 is advantageously calibrated to the position of user 102 who made the request 24. The audio data characterizing the spoken utterance of the request 24 can be captured by the microphone 20 of the vehicle 10 and indicate the respective location 310 of user 102 within the passenger compartment 300 of the vehicle 10.The modulator 220 then generates the playback sound field 302 such that the playback sound field 302 is centered on the respective location 310 of the user 102 in the passenger compartment 300. Fig. 5 shows a flowchart of an exemplary sequence of operations for a method 500 for sonifying navigation search results. The method 500 can be described with reference to Fig. 1-4, wherein the operations of the method 500 are performed while a user 102 is in a passenger compartment 300 of a vehicle 10. The data processing hardware (e.g., data processing hardware 12, 62 of Fig. 1) can execute instructions stored in the memory hardware (e.g., memory hardware 14, 64 of Fig. 1) to perform the exemplary sequence of operations for the method 500. In Operation 502, Procedure 500 comprises receiving a response 202 to a query 24 made by User 102. The response 202 contains a list of potential matches 204, 204a-n, where each potential match 204 in the list of potential matches 204 contains corresponding embedded metadata 206, 206a-n. For each potential match 204 in the list of potential matches 204, the procedure 500 also includes operations 504-508. In particular, in operation 504, the procedure 500 includes extracting the embedded metadata 206 corresponding to the potential match 204. In operation 506, the procedure 500 also includes determining, based on the extracted embedded metadata 206, a spatially arranged location 306 within a playback sound field 302 so that the user 102 can perceive it as the sound source 308 of the potential match 204. The procedure 500 further includes, in operation 508, playing back output audio signals 304 characterizing the potential match 203 through a loudspeaker arrangement 16 in the passenger compartment 300 to generate the playback sound field 302.In this process, the user 102 perceives the potential hit 204 as originating from the sound source 308 at the spatially arranged location 306 within the playback sound field 302. Several implementations have been described. However, it goes without saying that various modifications can be made without deviating from the spirit and scope of the disclosure. Accordingly, other embodiments also fall within the scope of protection of the following claims. The foregoing description serves for illustration and description purposes. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular configuration are generally not restricted to that particular configuration but are optionally interchangeable and may be used in a selected configuration even if they are not specifically shown or described. The same may also be varied in many ways. Such variations are not to be considered outside the scope of disclosure, and all such modifications are to be included within the scope of protection of the disclosure.

Claims

A computer-implemented method, executed in data processing hardware, that causes the data processing hardware to perform operations that include: while a user is in the passenger compartment of a vehicle: receiving audio data characterizing a spoken utterance of a request made by the user, recorded by a microphone in the passenger compartment of the vehicle, wherein the audio data characterizing the spoken utterance of the request indicates a specific location of the user in the passenger compartment of the vehicle; receiving a response to the request made by the user, wherein the response contains a list of potential matches to the request, each potential match in the list of potential matches containing corresponding embedded metadata; for each potential match in the list of potential matches: extracting the embedded metadata corresponding to the potential match;Determine, based on the extracted embedded metadata, a spatially located point within a playback sound field that the user can perceive as the sound source of the potential hit; and reproduce output audio signals characterizing the potential hit via a loudspeaker array in the passenger compartment to generate the playback sound field, wherein the user perceives the potential hit as emanating from the sound source at the spatially located point within the playback sound field, the playback sound field being centered on the determined location of the user in the passenger compartment of the vehicle. Method according to claim 1, wherein the operations further comprise: receiving location information of the vehicle. Method according to claim 2, wherein the reproduction of the output audio signals comprises modulating the output audio signals based on the distance between the location information of the vehicle and the embedded metadata of the potential hit. Method according to claim 1, wherein the operations further comprise: during the playback of the output audio signals characterizing the potential hit: simultaneous display of a corresponding visual cue on a graphical user interface of the vehicle. The method of claim 1, wherein the operations further comprise: identifying a user preference associated with one of the potential matches in the list of potential matches, wherein the user preference is stored in a user profile of the user. Method according to claim 5, wherein the reproduction of the output audio signals characterizing the potential match with the user preference comprises the reproduction of a user-defined audio output signal. Method according to claim 1, wherein the playback sound field represents locations outside the passenger compartment of the vehicle.

Images